Updates:
- 2025-02-27 – Corrected use of “hard links” to be “symbolic,” per William A. in the comments. Thanks!
NOTE: I am posting this as a plea for help. If you understand this problem, and know of solution, please leave a comment. I would really appreciate some input and clarification on my “understanding” of this issue.
TL:DNR: Symbolic links are a workaround.
The year is 2025, and open source projects can still be broken by having a space in a path name.
According to a robot, “Windows has allowed spaces in filenames and directory names since Windows NT 3.1, which was released in 1993.”
I have not fact checked this. My first PC was a Toshiba laptop I purchased at the end of 1995. (Not counting my DOS-based Tandy 1400LT I had years earlier.) At that point, it was Windows 3.1 that followed the 8.3 filename format that CP/M used.
FILENAME.TXT
It was the late Steve Bjork that convinced me to upgrade to the new Windows 95 when it came out. This upgrade allowed filenames up to 255 characters, if my quick web search is correct. I recall having a DOS C compiler (“Power C” – wow, they still sell this???) and it did not support the long filenames. Windows did provide some level of backwards compatibility by having any long filename still represented by a short name, which would have the first part of the filename then a “~1”, so something like “really long filename.txt” might also exist as “REALLY~1.TXT”.
Side note: There was a quirk with this implementation. Even filenames that could have fit in the 8 characters seemed to get this “~1” added to them. BUT, you could rename those to the normal 8 characters and is till worked. i.e. “filename.txt” would create “FILENA~1.TXT” but you could then “RENAME FILENA~1.TXT FILENAME.TXT” and it would show up as FILENAME.TXT. I actually wrote a C program that would go through the directory and rename any files like this. Fun times. And yikes, that was 30 years ago!?!?!
So basically, for three decades we could have spaces in filenames and directories on Windows.
Unix, back when it started, had short filenames and did not allow spaces initially. But, as long as I have been using it (my first exposure was 1995 on SunOS) I believe it did.
In the late 1980s I began using Microware’s OS-9 for the 6809 (on a Radio Shack Color Computer 3). While it had longer filenames than DOS (28 characters?), it did not allow spaces in filenames or directory names. I am unsure if the modern OS-9000 (renamed to just “OS-9 for x86” or whatever architecture, waybackwhen) has ever updated to allow spaces.
But I digress.
I still run across open source projects that fail if they are built from a path that has a space anywhere in it. For example, if you have this:
/usr/allenh/github downloads/…and you “git” your project there, the build might break because of that space. Scripts may just see it as “/usr/allenh/github” and fail. This seems to be the case for the NitrOS9 Project, a fantastic effort to recreate OS-9/6809 with tons of enhancements.
From my experience, I think these problems can be solved by simply adding quotes around paths in the build scripts. For example, if you have this:
DISCLAIMER: Just examples. I know about escaping spaces as “\ ” in *nix filesystems, and maybe something like that exists for Windows (let me know in the comment).
SOURCE_PATH=C:\Users\Allen Huffman\Development\Source Code
somecommand -xyz $(SOURCE_PATH$)
…that will fail because it ends up looking like three parameters after the options:
somecommand -xyz [C:\Users\Allen] [Huffman\Development\Source] [Code]
But if the build script “quotes” the file path, like this:
somecommand -xyz "$(SOURCE_PATH$)"
…then that gets passed to the command the same way:
somecommand -xyz "C:\Users\Allen Huffman\Development\Source Code"
And that seems to work.
But, again, this is 2025. I run into projects that fail with spaces in the path multiple times a year.
Just don’t put spaces, duh.
“Hey doctor, it hurts when I do this.”
“Then don’t do that.”
Why not just remove the spaces? In olden days, you were in control of directories because you made them. I came from a “no spaces allowed” OS background so I never used spaces. “SOURCECODE” or, if underscore was allowed, maybe “SOURCE_CODE” to be more readable (but I hate typing two extra keypresses to generate that underscore).
But, Windows may not agree. Once I did a fresh Windows install, and let it do all the defaults (Microsoft account to login, enable OneDrive, etc.) and the path it created had a space in it. Anything that I put in “my” home folder (Documents, etc.) had a space somewhere earlier in the path.
Likewise, on macOS, if you use iCloud, some default directories get links that make them look “normal”:
allenh@Mac Documents % pwd
/Users/allenh/Documents
…but if you decide to make your own directory on your “iCloud Drive”, you see the true long path — which has a space in it!
allenh@Mac Development % pwd
/Users/allenh/Library/Mobile Documents/com~apple~CloudDocs/Others/Development
That “Mobile Documents” path is the cause of so many problems.
My workaround was to simply move these folders outside of my self-created directory and put them into one of the defaults, like “~/Documents/Development”.
Symbolic links, anyone?
On macOS (a Unix-type operating system), and Linux (if such a problem exists there?), you can create symbolic-links to folders. You create a new name and then tell it where it points to.
ln -s /Users/allenh/Library/Mobile\ Documents/com\~apple\~CloudDocs/Others/Development ~/Development-iCloud
Now in my home folder, I have a directory called “Development-iCloud” that has no space in it, but points to the real spot with a space:
allenh@Mac ~ % cd Development-iCloud
allenh@Mac Development-iCloud % pwd
/Users/allenh/Development-iCloud
Now I can change into that folder and build scripts that were failing will work. Nice.
But what about Windows?
I have not tried it myself, mostly because I didn’t have A.I. to ask for a solution the last time I ran into this on Windows, but there is a “mklink.exe” command that appears similar, and may solve the problem there:
mklink /D "C:\path\to\link" "D:\path\to\actual"
It feels like this workaround should work there.
But is this what developers do? Most smart computer folks I know run away from anything cloud (and for good reason). I have been drinking the Kool-Aid for awhile and love having access to non-private files on all my devices, via syncing on Dropbox (mostly) or iCloud or OneDrive or Google Drive. I use all of them, for different purposes.
This is my workaround, but is there a better way?
Let me know in the comments.
Until then, stay off my lawn.
First note: in the text you mention “hard links to folders” then use “ln -s” which creates a symbolic link. You generally do NOT create hard links to folders on unix type systems. For those not in the know, a “hard link” is one where two directory entries refer to the exact same file physically on the disk. (Reference the same inode on Unix type file systems.) Symbolic links just encode a path to the real file name (relative or absolute). Hard links cannot go across file system boundaries and usually break things with directories, but symbolic links are safe for just about everything.
Now on to spaces in file names, even DOS, early Unix, and our “beloved” Disk Extended Color Basic allowed spaces in file names as long as you could get that file name into the relevant system calls. Command line tools didn’t really handle that well. And the way DOS, Windows, and OS9 pass command lines to programs, you don’t get a set of nicely delimited arguments in an array. You get the raw command line string. That means it is up to every program to handle quoting of arguments. (The C standard library and other aspects of the system may provide some higher level handling but at the core, it’s up to the program to figure it out.) Unix type systems actually pass an array of arguments and the shell or other calling program deals with quoting which makes it much more consistent.
Anyroad, tools like Make ended up with assumptions about spaces not being in file names and as a result their handling of stuff internally assumes lists and stuff can be broken on whitespace. You can work around that for some stuff like passing stuff to commands (quoting the arguments). However, Make really can’t handle spaces in any of the paths it has to directly process. That is why you should never, ever, reference the full path to anything in a Makefile. Don’t force expand the current working directory or anything like that. If you do, you run into problems like Allen has. If your Makefiles rely on absolute paths, even absolute paths automatically determined, you have almost certainly done something wrong and there is almost certainly another way that doesn’t do that. Basically, if you can keep everything relative, it won’t matter if there are spaces in the file path further up the tree.
Of course, none of that helps when you have to build a third party software package that does encode absolute paths into the build process.
Ah, I never understood hard versus symbolic links. Makes sense. Thank you!
I also did not recall you could put spaces in the DOS files, though I guess you could shove anything you want in a directory. OS-9 can’t do that, can it? I mean, sure, you could put them in the entry, but how would you get to them? Do the RBF calls have a way?
And for the makefiles, I did not think there could be any full paths in the NitrOS-9 makefiles, but I guess there must be — there is an export you do to set the location of the files, and that is creating a full paths. I wonder if there is a way to put a relative path in that export.
Unix makes a distinction between soft and hard links. Very early versions of Unix (at least as described in The Bell System Technical Journal, July-August 1978) allowed arbitrary hard links for directories, but was soon disallowed because it could turn the filesystem from a hierarchy to a directed graph with loops that would play havoc with file system tools. You can still have hard links to files.
In William’s description, he mentioned one could point to a different filesystem, which makes me think it was like how you could mount a file system/directory (unsure of the terminology) and it showed up as if it was a directory. Is that closer to what a hard link is?
If you do it with a file, and link “test.txt” somewhere else, editing the original or the linked copy act on the same copy (only one). What does hard linking do?
Everything stored on a disk has an inode. An inode is a block of information about a file, where the data for the file is on the disk, various time stamps, permissions, size, type, but NOT THE NAME! The
typefield indicates if the file is just a plain file, symbolic link or a directory (with a few others that aren’t important right now). A symbolic link is just a file whose contents is a path to another file that the kernel uses. A directory is just a file that maps names to inodes and is another file type the kernel uses. There are system calls to open and parse a directory to isolate a program from having to know the underlying file system.When you create a file on Unix, say “foo”, then an entry in the directory is created that maps the name “foo” with an inode. That is a “hard link”. You can create another such hard link with a new name, say “bar”. It doesn’t have to be in the same directory as “foo”, but it does have to be on the same disk partition, as inodes are only unique per partition (since a soft link is just a path, it can point to any file in the filesystem, independent of any partitions). So for example, if “/usr/myself/foo” points to inode 1000, you can create “/tmp/bar” that also points to inode 1000. If you edit “/usr/myself/foo”, you are also editing “/tmp/bar” since they both point to the same thing on the disk. The inode also tracks the number of hard links it has, so it knows when to actually delete a file (when the hard link number is 0). You can see how many hard links a file has with the command
ls -l:-rw-r--r-- 2 spc spc 0 Feb 28 17:30 barThat initial 2 is the hard link count. Good luck finding the other hard link on the file system (the 0 is the size of the file by the way).
It used to be that you could hard link a directory, but that was found to be too problematic and not allowed (except for “..”, which is a name given to the parent directory and is managed by the kernel).
Mounting a partition is something different. To mount a partition (or disk), you need to tell Unix which partition to use (a device name), and an existing directory entry to mount it to. A directory has to exist on some partition before you can mount it (the root partition is the only exception here) and any contents in the directory are no longer visible when a file system is mounted over it (except for “union file systems” but that’s beyond the scope of this comment).
Worth knowing is that AFAIK at least in Linux the term “iname” is used both to represent that metadata on disk but also in memory. I kind of get why that happened, but it makes it confusing trying to understand how the file system code works.
In most other OSes whatever is on disk have different names from what is in memory. Typically you would have a handle in memory, and what’s on disk might not have a name or might have a more complicated name. (Other names for the in-memory things exist too, for example FCB for CP/M and early MS-/PC-DOS).
Interesting. On my web host, I know it now (didn’t use to) has a limit to “inodes” that my photo galleries had exceeded, so I had to split them up into multiple accounts. I never understood what an inode was and why they just didn’t say “files.” Thanks for this! That helps greatly.
Thank you. Spaces in a filename are evil. I use camel case.
The classic OS-9 convention was to use UPPERCASE for directories, and lowercase for files, and I’ve been stuck in the habit for a long time. But I do find myself, on Mac, putting spaces in filenames these days. :(
Question: camelCase, or PascalCase?
Wouldn’t it be dromedarCase and CamelCase? :)
You folks need to stop making me look things up ;-) Well done.
I’ve found over the years that file systems can often accept more characters than one would expect. I know you can create spaces in files for the Coco, and definitely for Unix. MS-DOS, and Windows, can use ‘/’ instead of ‘\’ in paths even! It’s the shells that cause issues. Unix shells break the command line up by spaces, so spaces in a file name are problematic at the shell.
COMMAND.COM(of MS-DOS) doesn’t like using ‘/’ for paths.As for a workaround, I once suggested using the non-breaking space from Unicode, but that idea wasn’t well received. I don’t know why, it seemed logical to me.
A nbsp character would be quite useful. I had forgotten about spaces on RS-DOS, but I used them — “RUN ME.BAS” and such. Also, making files with CHR$(128+) — especially 143, which I think was the block the same color as the 32-column screen, so you had files people saw in a DIR but could not load/kill/run if they didn’t know where that character was.
CoCo 3 40/80 made those show up as weird characters, spoiling that.
I need to boot a DOSBOX and play with files. For some reason, I had it in my head that nothing allowed spaces but things like Unix and Mac OS, back then, and even the CoCo had them… Except the OS-9 stuff I am unclear on. It sounds like it could, but you’d have to parse it yourself because of how the RBF calls worked. I need to dig into that.
In Windows I think that at least one of the join/subst/assign/whatever tools, whichever survives since the old DOS days, can be used to access a directory with spaces from a non-space “nickname”.
You can also as a work-around share a “problematic path” and locally map it as a network share. (This can also be a trick to “hide” files on a computer. Create a directory, and then create a directory within the first directory. Share the last directory, and then set the permissions for the first directory so no user have access to it. Of course anyone looking for this will find it, but it might hide files from say certain scripts and whatnot).
A classic Windows space-in-path problem was when Windows 95 were new, and some developers hard coded the path to where to install programs and also failed to put quotation marks around the full path name for their application.
The result was that in Swedish Windows 95 the default folder for programs was called “C:\Program”, and things worked nicely until some program installed in the English language default path “C:\Program Files” and also didn’t put quotation marks. The result was that when Windows was told to start C:\Program Files\Whatever\Whatever.exe it would interpret it as to start C:\Program with Files\Whatever\Whatever.exe an argument. With programs intending to auto start, the result was that Windows would just open an explorer window showing the Program folder.
Btw it was just a few years ago I tried to report this as a bug in the installer for an application to use Microbee (the Australian Z80 based computer from the 80’s) disks in a modern Windows computer, but let’s just say that the response I got (in a thread in the Microbee-something forum) gave me the impression that the creator of that application seems like someone I’m not keen on interacting with.
Btw re file names and whatnot in Windows:
Using non-Windows / non-Microsoft SMB clients it’s possible to create file names that end in a space or a dot, and those can be really tricky to rename within Windows. This typically happens when downloading files on a computer that allows semi-long files (IIRC around 30 characters on an Amiga) and also don’t interpret spaces or dots as any special character. Too long file names sometimes ends up truncated in a way that they end with a dot or space, and when copying those files to a Windows share you may end up with those file names.
Hmm, SMB must have had some kind of provision for file name massaging when it was adopted by different file system OSes. Like, “file name.txt” changing to “file_name.txt”. Not sure what Microware’s OS-9 did on that, but I know it has SMB and such (not the 6809 version).
What was the Microbee disk bug?
I’ve never seen SMB change spaces to underscores?
The Microbee utility (modern enthusiast third party) was that it ignored the Windows registry entry for where programs are to be stored and used the hard coded “Program Files”, but also omitted spaces along the path+file name to the program.