Pete Freitag Pete Freitag

The newline cat mystery

Updated on November 13, 2023
By Pete Freitag

I ran into a really strange problem today, whenever I would write a file it would show up as empty on my file system.

Here's a simplified version of my code:

var nl = chr(13);
var csv = '"order_id","date"' & nl;
csv &= '"1","2023-01-01"' & nl;
fileWrite("/tmp/test.txt", csv);

I would then go to my terminal and type:

cat /tmp/test.txt

And it would not output anything, it appeared to be an empty file!

As a sanity check I added a throw statement to the end of my code which read the file content.

var nl = chr(13);
var csv = '"order_id","date"' & nl;
csv &= '"1","2023-01-01"' & nl;
fileWrite("/tmp/test.txt", csv);
throw ( fileRead("/tmp/test.txt") );

Unfortunately this did not create any sanity because it would throw the contents of the file, but cat was saying that it was empty.

Now one thing that I should have noticed was that the file size was not zero bytes, so the file was actually not empty.

Cat is sneaky, and I used the wrong character code for newline

The problem was that I was using chr(13) (carriage return) where I meant to use chr(10) (new line). I've made this kind of mistake enough times to write a really detailed blog entry called What's the difference between ASCII Chr(10) and Chr(13)?, but this one still managed to baffle me for a while.

The other problem was that cat was not outputting anything in this case even though there was stuff in the file. It turns out that you can use cat -v to show Display non-printing characters, and then it will actually output:


Still I find it very odd that cat didn't show anything, perhaps it is because the carriage return, brings the cursor back to the beginning of the line, effectively erasing the line that was just output. The moral of the story that cat will sometimes hide stuff from you, so you might need to check your file contents another way.


As some have confirmed, the carriage return does indeed bring the cursor back to the beginning of the terminal, and will overwrite what was already printed. Here's an example that illustrates this:

echo "sanity\rodd" | cat

Which will output:


Now the reason I was not seeing anything is because the line I was outputting was shorter than my terminal prompt, so it overwrote it.


The newline cat mystery was first published on November 08, 2023.


The Fixinator Code Security Scanner for ColdFusion & CFML is an easy to use security tool that every CF developer can use. It can also easily integrate into CI for automatic scanning on every commit.

Try Fixinator

The weekly newsletter for the CFML Community


I believe that this is the recommended method to identify the system line separator regardless of platform/version used.

lineSeparator = createobject("java", "java.lang.System").getProperty("line.Separator");

NOTE: I found all java system settings referenced in one of your blog entries from 2004.
by James Moberg on 11/10/2023 at 5:59:55 PM UTC
I started out when terminals had printheads that printed and then moved with each character printed. The carriage return moved the printhead back to the beginning of the line, the linefeed moved the paper up one line. When the first CRT terminals came in the cursor took the place of the printhead and the character would be written where . The CR moved the cursor back to the start-of-line and the FF moved the cursor to the next line. Since a CRT terminal can't do overstrikes (which is what would happen on a paper terminal) the attempt to write on a line after a CR without an following FF would clear the line. So this is not a weirdness with `cat`, it is a weirdness in how your virtual terminal interprets the sequence of bytes that cat is feeding it from your file. If you had a very slow computer you might even see the cursor put the characters up and then have them disappear.
by verisimilidude on 11/12/2023 at 7:47:44 PM UTC
@James - thanks, that method does give you a CRLF on Windows servers and a LF on Linux, Mac or everything else. I find that many files just use the LF chr(10) character, and these days most windows apps can deal with it just fine. So while I used to use that method quite a bit, as of late I tend to just use the LF / new line character code (sometimes the CR for fun :-). Though your method certainly would have avoided this problem!

@verisimilidude - thanks for sharing your experiences with a printhead! I have been experimenting a bit more with this since I posted it and I will probably give some updates with more info / explanation. I do agree cat is not wrong, or weird it was me :-)
by Pete Freitag on 11/13/2023 at 3:14:56 PM UTC

Post a Comment