newlib/winsup/doc/specialnames.xml

<?xml version="1.0" encoding='UTF-8'?>
<!DOCTYPE sect1 PUBLIC "-//OASIS//DTD DocBook V4.5//EN"
		"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">

<sect1 id="using-specialnames"><title>Special filenames</title>

<sect2 id="pathnames-etc"><title>Special files in /etc</title>

<para>Certain files in Cygwin's <filename>/etc</filename> directory are
read by Cygwin before the mount table has been established.  The list
of files is</para>

<screen>
  /etc/fstab
  /etc/fstab.d/$USER
  /etc/passwd
  /etc/group
</screen>

<para>These file are read using native Windows NT functions which have
no notion of Cygwin symlinks or POSIX paths.  For that reason
there are a few requirements as far as <filename>/etc</filename> is
concerned.</para>

<para>To access these files, the Cygwin DLL evaluates it's own full
Windows path, strips off the innermost directory component and adds
"\etc".  Let's assume the Cygwin DLL is installed as
<filename>C:\cygwin\bin\cygwin1.dll</filename>.  First the DLL name as
well as the innermost directory (<filename>bin</filename>) is stripped
off: <filename>C:\cygwin\</filename>.  Then "etc" and the filename to
look for is attached: <filename>C:\cygwin\etc\fstab</filename>.  So the
/etc directory must be parallel to the directory in which the cygwin1.dll
exists and <filename>/etc</filename> must not be a Cygwin symlink
pointing to another directory.  Consequentially none of the files from
the above list, including the directory <filename>/etc/fstab.d</filename>
is allowed to be a Cygwin symlink either.</para>

<para>However, native NTFS symlinks and reparse points are transparent
when accessing the above files so all these files as well as
<filename>/etc</filename> itself may be NTFS symlinks.</para>

<para>Last but not least, make sure that these files are world-readable.
Every process of any user account has to read these files potentially,
so world-readability is essential.  The only exception are the user
specific files <filename>/etc/fstab.d/$USER</filename>, which only have
to be readable by the $USER user account itself.</para>

</sect2>

<sect2 id="pathnames-dosdevices"><title>Invalid filenames</title>

<para>Filenames invalid under Win32 are not necessarily invalid under Cygwin.
There are a few rules which apply to Windows filenames.  Most notably, DOS
device names like <filename>AUX</filename>, <filename>COM1</filename>,
<filename>LPT1</filename> or <filename>PRN</filename> (to name a few)
cannot be used as filename or extension in a native Win32 application.
So filenames like <filename>prn.txt</filename> or <filename>foo.aux</filename>
are invalid filenames for native Win32 applications.</para>

<para>This restriction doesn't apply to Cygwin applications.  Cygwin
can create and access files with such names just fine.  Just don't try
to use these files with native Win32 applications.</para>

</sect2>

<sect2 id="pathnames-specialchars">
<title>Forbidden characters in filenames</title>

<para>Some characters are disallowed in filenames on Windows filesystems.
These forbidden characters are the ASCII control characters from ASCII
value 1 to 31, plus the following characters which have a special meaning
in the Win32 API:</para>

<screen>
  "   *   :   &lt;   &gt;   ?   |   \
</screen>

<para>Cygwin can't fix this, but it has a method to workaround this
restriction.  All of the above characters, except for the backslash,
are converted to special UNICODE characters in the range 0xf000 to 0xf0ff
(the "Private use area") when creating or accessing files by adding 0xf000
to the forbidden characters' code points.</para>

<para>The backslash has to be exempt from this conversion, because Cygwin
accepts Win32 filenames including backslashes as path separators on input. 
Converting backslashes using the above method would make this impossible.</para>

<para>Additionally Win32 filenames can't contain trailing dots and spaces
for DOS backward compatibility.  When trying to create files with trailing
dots or spaces, all of them are removed before the file is created.  This
restriction only affects native Win32 applications.  Cygwin applications
can create and access files with trailing dots and spaces without problems.
</para>

<para>An exception from this rule are some network filesystems (NetApp,
NWFS) which choke on these filenames.  They return with an error like
"No such file or directory" when trying to create such files.  Cygwin
recognizes these filesystems and works around this problem by applying
the same rule as for the other forbidden characters.  Leading spaces and
trailing dots and spaces will be converted to UNICODE characters in the
private use area.  This behaviour can be switched on explicitely for a
filesystem or a directory tree by using the mount option
<literal>dos</literal>.</para>

</sect2>

<sect2 id="pathnames-unusual">
<title>Filenames with unusual (foreign) characters</title>

<para> Windows filesystems use Unicode encoded as UTF-16
to store filename information.  If you don't use the UTF-8
character set (see <xref linkend="setup-locale"></xref>) then there's a
chance that a filename is using one or more characters which have no
representation in the character set you're using.</para>

<note><para>In the default "C" locale, Cygwin creates filenames using
the UTF-8 charset.  This will always result in some valid filename by
default, but again might impose problems when switching to a non-"C"
or non-"UTF-8" charset.</para></note>

<note><para>To avoid this scenario altogether, always use UTF-8 as the
character set.</para></note>

<para>If you don't want or can't use UTF-8 as character set for whatever
reason, you will nevertheless be able to access the file.  How does that
work?  When Cygwin converts the filename from UTF-16 to your character
set, it recognizes characters which can't be converted.  If that occurs,
Cygwin replaces the non-convertible character with a special character
sequence.  The sequence starts with an ASCII CAN character (hex code
0x18, equivalent Control-X), followed by the UTF-8 representation of the
character.  The result is a filename containing some ugly looking
characters.  While it doesn't <emphasis role='bold'>look</emphasis> nice, it
<emphasis role='bold'>is</emphasis> nice, because Cygwin knows how to convert
this filename back to UTF-16.  The filename will be converted using your
usual character set.  However, when Cygwin recognizes an ASCII CAN
character, it skips over the ASCII CAN and handles the following bytes as
a UTF-8 character.  Thus, the filename is symmetrically converted back to
UTF-16 and you can access the file.</para>

<note><para>Please be aware that this method is not entirely foolproof.
In some character set combinations it might not work for certain native
characters.</para>

<para>Only by using the UTF-8 charset you can avoid this problem safely.
</para></note>

</sect2>

<sect2 id="pathnames-casesensitive">
<title>Case sensitive filenames</title>

<para>In the Win32 subsystem filenames are only case-preserved, but not
case-sensitive.  You can't access two files in the same directory which
only differ by case, like <filename>Abc</filename> and
<filename>aBc</filename>.  While NTFS (and some remote filesystems)
support case-sensitivity, the NT kernel does not support it by default.
Rather, you have to tweak a registry setting and reboot.  For that reason,
case-sensitivity can not be supported by Cygwin, unless you change that
registry value.</para>

<para>If you really want case-sensitivity in Cygwin, you can switch it
on by setting the registry value</para>

<screen>
HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\kernel\obcaseinsensitive
</screen>

<para>to 0 and reboot the machine.</para>

<note>
<para>
When installing Microsoft's Services For Unix (SFU), you're asked if
you want to use case-sensitive filenames.  If you answer "yes" at this point,
the installer will change the aforementioned registry value to 0, too.  So, if
you have SFU installed, there's some chance that the registry value is already
set to case sensitivity.
</para>
</note>

<para>After you set this registry value to 0, Cygwin will be case-sensitive
by default on NTFS and NFS filesystems.  However, there are limitations: 
while two <emphasis role='bold'>programs</emphasis> <filename>Abc.exe</filename>
and <filename>aBc.exe</filename> can be created and accessed like other files,
starting applications is still case-insensitive due to Windows limitations
and so the program you try to launch may not be the one actually started.  Also,
be aware that using two filenames which only differ by case might
result in some weird interoperability issues with native Win32 applications.  
You're using case-sensitivity at your own risk.  You have been warned! </para>

<para>Even if you use case-sensitivity, it might be feasible to switch to
case-insensitivity for certain paths for better interoperability with
native Win32 applications (even if it's just Windows Explorer).  You can do
this on a per-mount point base, by using the "posix=0" mount option in
<filename>/etc/fstab</filename>, or your <filename>/etc/fstab.d/$USER</filename>
file.</para>

<para><filename>/cygdrive</filename> paths are case-insensitive by default.
The reason is that the native Windows %PATH% environment variable is not
always using the correct case for all paths in it.  As a result, if you use
case-sensitivity on the <filename>/cygdrive</filename> prefix, your shell
might claim that it can't find Windows commands like <command>attrib</command>
or <command>net</command>.  To ease the pain, the <filename>/cygdrive</filename>
path is case-insensitive by default and you have to use the "posix=1" setting
explicitly in <filename>/etc/fstab</filename> or
<filename>/etc/fstab.d/$USER</filename> to switch it to case-sensitivity,
or you have to make sure that the native Win32 %PATH% environment variable
is using the correct case for all paths throughout.</para>

<para>Note that mount points as well as device names and virtual
paths like /proc are always case-sensitive!  The only exception are
the subdirectories and filenames under /proc/registry, /proc/registry32
and /proc/registry64.  Registry access is always case-insensitive.
Read on for more information.</para>

</sect2>

<sect2 id="pathnames-casesensitivedirs">
<title>Case sensitive directories</title>

<para>Windows 10 1803 introduced a new feature: NTFS directories can be marked
as case-sensitive, independently of the <literal>obcaseinsensitive</literal>
registry key discussed in the previous section.  This new per-directory
case-sensitivity requires setting a flag in the NTFS filesystem header which
is, unfortunately, undocumented.  The result is that you have to activate
<literal>Windows Subsystem for Linux</literal> (<literal>WSL</literal>), a
feature available via <literal>Programs and Features</literal> ->
<literal>Turn Windows features on or off</literal>.  You only have to activate
<literal>WSL</literal>, you don't have to install any actual Linux.  After
turning <literal>WSL</literal> on and performing the compulsory reboot,
case-sensitive directories are activated.</para>

<para>Of course, there's a drawback.  While these case-sensitive directories
work like charm on the local machine, there are massive interoperability
problems when trying to access these directories from remote machines at
the time of writing this.  We opened a bug report for that at
<ulink url="https://github.com/Microsoft/WSL/issues/3885">Microsoft's WSL issue tracker</ulink>,
if you're interested in the details.</para>

<note><para>If you want case-sensitivity and need interoperability with remote
machines, better stick to switching the kernel to case-sensitivity as
outlined in <xref linkend="pathnames-casesensitive"></xref></para></note>

<para>With <literal>WSL</literal> activated and starting with Cygwin 3.0.0,
Cygwin's <command>mkdir</command> system call automatically created all
directories below the Cygwin installation directory as case-sensitive.
With Cygwin 3.0.2, this feature had been disabled again for hopefully
obvious reasons.</para>

<para>However, you can still use Cygwin's new
<xref linkend="chattr"></xref> tool with the <literal>-C</literal> option
to control case-sensitivity of directories on NTFS filesystems.</para>

<para>Please keep in mind that switching <emphasis>off</emphasis>
case-sensitivity on a directory has a condition attached to it:  If
the directory contains two files which only differ in case (e. g.,
<filename>foo</filename> and <filename>FOO</filename>), Windows
refuses to convert the dir back to case-insensitive.  First you have
to fix the filename collision, i. e., you have to rename one of these
files.</para>

</sect2>

<sect2 id="pathnames-posixdevices"> <title>POSIX devices</title>
<para>While there is no need to create a POSIX <filename>/dev</filename> 
directory, the directory is automatically created as part of a Cygwin
installation.  It's existence is often a prerequisit to run certain
applications which create symbolic links, fifos, or UNIX sockets in
<filename>/dev</filename>.  Also, the directories <filename>/dev/shm</filename>
and <filename>/dev/mqueue</filename> are required to exist to use named POSIX
semaphores, shared memory, and message queues, so a system without a real
<filename>/dev</filename> directory is functionally crippled.
</para>

<para>Apart from that, Cygwin automatically simulates POSIX devices
internally.  The <filename>/dev</filename> directory is automagically
populated with existing POSIX devices by Cygwin in a way comparable with a
<ulink url="http://en.wikipedia.org/wiki/Udev">udev</ulink> based virtual
<filename>/dev</filename> directory under Linux.</para>

<para>
Cygwin supports the following character devices commonly found on POSIX systems:
</para>

<screen>
/dev/null
/dev/zero
/dev/full

/dev/console	Pseudo device name for the current console window of a session.
		Cygwin's /dev/console is not quite comparable with the console
		device on UNIX machines.

/dev/cons0      Console sessions are numbered from  /dev/cons0 upwards.
/dev/cons1	Console device names are pseudo device names, only accessible
...		from processes within this very console session.  This is due
		to a restriction in Windows.

/dev/tty	The current controlling tty of a session.

/dev/ptmx	Pseudo tty master device.

/dev/pty0	Pseudo ttys are numbered from /dev/pty0 upwards as they are
/dev/pty1	requested.
...

/dev/ttyS0	Serial communication devices.  ttyS0 == Win32 COM1,
/dev/ttyS1	ttyS1 == COM2, etc.
...

/dev/pipe
/dev/fifo

/dev/kmsg	Kernel message pipe, for usage with sys logger services.

/dev/random	Random number generator.
/dev/urandom

/dev/dsp	Default sound device of the system.
</screen>

<para>
Cygwin also has several Windows-specific devices:
</para>

<screen>
/dev/com1	The serial ports, starting with COM1 which is the same as ttyS0.
/dev/com2	Please use /dev/ttySx instead.
...

/dev/conin	Same as Windows CONIN$.
/dev/conout	Same as Windows CONOUT$.
/dev/clipboard	The Windows clipboard, text only
/dev/windows	The Windows message queue.
</screen>

<para>
Block devices are accessible by Cygwin processes using fixed POSIX device
names.  These POSIX device names are generated using a direct conversion
from the POSIX namespace to the internal NT namespace.
E.g. the first harddisk is the NT internal device \device\harddisk0\partition0
or the first partition on the third harddisk is \device\harddisk2\partition1.
The first floppy in the system is \device\floppy0, the first CD-ROM is
\device\cdrom0 and the first tape drive is \device\tape0.</para>

<para>The mapping from physical device to the name of the device in the
internal NT namespace can be found in various places.  For hard disks and
CD/DVD drives, the Windows "Disk Management" utility (part of the
"Computer Management" console) shows that the mapping of "Disk 0" is
\device\harddisk0.  "CD-ROM 2" is \device\cdrom2.  Another place to find
this mapping is the "Device Management" console.  Disks have a
"Location" number, tapes have a "Tape Symbolic Name", etc.
Unfortunately, the places where this information is found is not very
well-defined.</para>

<para>
For external disks (USB-drives, CF-cards in a cardreader, etc) you can use
Cygwin to show the mapping.  <filename>/proc/partitions</filename>
contains a list of raw drives known to Cygwin.  The <command>df</command>
command shows a list of drives and their respective sizes.  If you match
the information between <filename>/proc/partitions</filename> and the
<command>df</command> output, you should be able to figure out which
external drive corresponds to which raw disk device name.</para>

<note><para>Apart from tape devices which are not block devices and are
by default accessed directly, accessing mass storage devices raw
is something you should only do if you know what you're doing and know how to
handle the information.  <emphasis role='bold'>Writing</emphasis> to a raw
mass storage device you should only do if you
<emphasis role='bold'>really</emphasis> know what you're doing and are aware
of the fact that any mistake can destroy important information, for the
device, and for you.  So, please, handle this ability with care.
<emphasis role='bold'>You have been warned.</emphasis></para></note>

<para>
Last but not least, the mapping from POSIX /dev namespace to internal
NT namespace is as follows:
</para>

<screen>
POSIX device name     Internal NT device name

/dev/st0	      \device\tape0, rewind
/dev/nst0	      \device\tape0, no-rewind
/dev/st1	      \device\tape1
/dev/nst1	      \device\tape1
...
/dev/st15
/dev/nst15

/dev/fd0	      \device\floppy0
/dev/fd1	      \device\floppy1
...
/dev/fd15

/dev/sr0	      \device\cdrom0
/dev/sr1	      \device\cdrom1
...
/dev/sr15

/dev/scd0	      \device\cdrom0
/dev/scd1	      \device\cdrom1
...
/dev/scd15

/dev/sda	      \device\harddisk0\partition0	(whole disk)
/dev/sda1	      \device\harddisk0\partition1	(first partition)
...
/dev/sda15	      \device\harddisk0\partition15	(fifteenth partition)

/dev/sdb	      \device\harddisk1\partition0
/dev/sdb1	      \device\harddisk1\partition1

[up to]

/dev/sddx	      \device\harddisk127\partition0
/dev/sddx1	      \device\harddisk127\partition1
...
/dev/sddx15	      \device\harddisk127\partition15
</screen>

<para>
if you don't like these device names, feel free to create symbolic
links as they are created on Linux systems for convenience:
</para>

<screen>
ln -s /dev/sr0 /dev/cdrom
ln -s /dev/nst0 /dev/tape
...
</screen>

</sect2>

<sect2 id="pathnames-exe"><title>The .exe extension</title>

<para>Win32 executable filenames end with <filename>.exe</filename>
but the <filename>.exe</filename> need not be included in the command,
so that traditional UNIX names can be used.  However, for programs that
end in <filename>.bat</filename> and <filename>.com</filename>, you
cannot omit the extension.  </para>

<para>As a side effect, the <command> ls filename</command> gives
information about <filename>filename.exe</filename> if
<filename>filename.exe</filename> exists and <filename>filename</filename>
does not.  In the same situation the function call
<function>stat("filename",..)</function> gives information about
<filename>filename.exe</filename>.  The two files can be distinguished
by examining their inodes, as demonstrated below.
<screen>
<prompt>bash$</prompt> <userinput>ls * </userinput>
a      a.exe     b.exe
<prompt>bash$</prompt> <userinput>ls -i a a.exe</userinput>
445885548 a       435996602 a.exe
<prompt>bash$</prompt> <userinput>ls -i b b.exe</userinput>
432961010 b       432961010 b.exe
</screen>
If a shell script <filename>myprog</filename> and a program
<filename>myprog.exe</filename> coexist in a directory, the shell
script has precedence and is selected for execution of
<command>myprog</command>.  Note that this was quite the reverse up to
Cygwin 1.5.19.  It has been changed for consistency with the rest of Cygwin.
</para>

<para>The <command>gcc</command> compiler produces an executable named
<filename>filename.exe</filename> when asked to produce
<filename>filename</filename>. This allows many makefiles written
for UNIX systems to work well under Cygwin.</para>

</sect2>

<sect2 id="pathnames-proc"><title>The /proc filesystem</title> 
<para>
Cygwin, like Linux and other similar operating systems, supports the
<filename>/proc</filename> virtual filesystem. The files in this
directory are representations of various aspects of your system,
for example the command <userinput>cat /proc/cpuinfo</userinput> 
displays information such as what model and speed processor you have.
</para>
<para>
One unique aspect of the Cygwin <filename>/proc</filename> filesystem
is <filename>/proc/registry</filename>, see next section.
</para>
<para>
The Cygwin <filename>/proc</filename> is not as complete as the
one in Linux, but it provides significant capabilities. The
<systemitem>procps</systemitem> package contains several utilities
that use it.
</para>
</sect2>

<sect2 id="pathnames-proc-registry"><title>The /proc/registry filesystem</title>
<para>
The <filename>/proc/registry</filename> filesystem provides read-only
access to the Windows registry.  It displays each <literal>KEY</literal>
as a directory and each <literal>VALUE</literal> as a file.  As anytime
you deal with the Windows registry, use caution since changes may result
in an unstable or broken system.  There are additionally subdirectories called
<filename>/proc/registry32</filename> and <filename>/proc/registry64</filename>.
They are identical to <filename>/proc/registry</filename> on 32 bit
host OSes.  On 64 bit host OSes, <filename>/proc/registry32</filename>
opens the 32 bit processes view on the registry, while
<filename>/proc/registry64</filename> opens the 64 bit processes view.
</para>
<para>
Reserved characters ('/', '\', ':', and '%') or reserved names
(<filename>.</filename> and <filename>..</filename>) are converted by
percent-encoding:
<screen>
<prompt>bash$</prompt> <userinput>regtool list -v '\HKEY_LOCAL_MACHINE\SYSTEM\MountedDevices'</userinput>
...
\DosDevices\C: (REG_BINARY) = cf a8 97 e8 00 08 fe f7
...
<prompt>bash$</prompt> <userinput>cd /proc/registry/HKEY_LOCAL_MACHINE/SYSTEM</userinput>
<prompt>bash$</prompt> <userinput>ls -l MountedDevices</userinput>
...
-r--r----- 1 Admin SYSTEM  12 Dec 10 11:20 %5CDosDevices%5CC%3A
...
<prompt>bash$</prompt> <userinput>od -t x1 MountedDevices/%5CDosDevices%5CC%3A</userinput>
0000000 cf a8 97 e8 00 08 fe f7 01 00 00 00
</screen>
The unnamed (default) value of a key can be accessed using the filename
<filename>@</filename>.
</para>
<para>
If a registry key contains a subkey and a value with the same name
<filename>foo</filename>, Cygwin displays the subkey as
<filename>foo</filename> and the value as <filename>foo%val</filename>.
</para>
</sect2>

<sect2 id="pathnames-at"><title>The @pathnames</title> 
<para>To circumvent the limitations on shell line length in the native
Windows command shells, Cygwin programs, when invoked by non-Cygwin processes, expand their arguments
starting with "@" in a special way.  If a file
<filename>pathname</filename> exists, the argument
<filename>@pathname</filename> expands recursively to the content of
<filename>pathname</filename>. Double quotes can be used inside the
file to delimit strings containing blank space. 
In the following example compare the behaviors
<command>/bin/echo</command> when run from bash and from the Windows command prompt.</para>

<example id="pathnames-at-ex"><title> Using @pathname</title>
<screen>
<prompt>bash$</prompt> <userinput>/bin/echo  'This   is   "a     long"  line' > mylist</userinput>
<prompt>bash$</prompt> <userinput>/bin/echo @mylist</userinput>
@mylist
<prompt>bash$</prompt> <userinput>cmd</userinput>
<prompt>c:\&gt;</prompt> <userinput>c:\cygwin\bin\echo @mylist</userinput>
This is a     long line
</screen>
</example>
</sect2> 
</sect1>