4

I'm using Vim 8.2 to edit some text files for a very old MSX-DOS system; these have a Ctrl-Z (0x1A) character after the final CR LF to indicate the end of the file, as is usual with CP/M and similar disk operating systems.

I normally edit with fileformats=unix and nofixendofline and in this case I see, after the last line of text and trailing <0d>, a line with the Ctrl-Z character, displayed as <1a>. I can edit and save and, so long as I manually add the CR at the end of any new lines I add, all is fine.

When I switch to DOS fileformat and re-read the file (:set fileformats=dos | e) the <0d> characters disappear (as expected), and so does the final <1a> character. But unlike the CRs, the Ctrl-Z is not added back in when I write out the file. (I'm writing it immediately after the above command, without changing anything.)

Is there a way when using fileformat=dos to get Vim to keep the Ctrl-Z at EOF? I searched the help brought up by :help fileformats (the help file is options.txt) which says:

When "dos" is used, CTRL-Z at the end of a file is ignored.  
See |file-formats| and |file-read|.

*file-read* (in insert.txt) says the same.

4
  • fileformats is global option used for auto-detection. It defaults to unix,dos and should never be touched. If ever need to force format only set the local fileformat. E.g. e ++ff=dos
    – Matt
    Oct 19, 2022 at 6:58
  • @Matt It does not default to unix,dos on every platform. And if it really never should be touched, it would not be settable, much less have options that are not in the default list. ¶ I happen prefer always initially to use Unix format, even on Windows, and even for DOS files (because I don't like Vim silently deleting characters from files as I load them, as well as because it serves as a good, obvious warning that I'm editing DOS-format files). Thus I have fileformats=unix as my default, and I switch to fileformats=unix,dos and re-read the file when I wish to enable DOS-mode editing.
    – cjs
    Oct 19, 2022 at 8:45
  • 1. If you share config for multiple platforms then do set ffs=unix,dos once in vimrc 2. Setting ffs=unix breaks auto-detection, so it's a wrong thing to do. 3. As "unix" precedes "dos" in ffs, the default for new files will be "unix" anyway. 4. Then manually setting fileformat would only be required one out of 100 times if auto-detection ever fails.
    – Matt
    Oct 19, 2022 at 10:34
  • @Matt Breaking auto-detection is the wrong thing for you. You are not aware of environments in which I work, so that's not a decision you can or should make for me. And manually switching to DOS mode is trivial with my Vim configuration: two keystrokes.
    – cjs
    Oct 19, 2022 at 15:46

3 Answers 3

3

Vim unconditionally removes the ^Z unless 'binary' is set, but then Vim always uses Unix line endings. See readfile() in fileio.c:

/*
 * If we get EOF in the middle of a line, note the fact and
 * complete the line ourselves.
 * In Dos format ignore a trailing CTRL-Z, unless 'binary' set.
 */
if (!error
    && !got_int
    && linerest != 0
    && !(!curbuf->b_p_bin
        && fileformat == EOL_DOS
        && *line_start == Ctrl_Z
        && ptr == line_start + 1))
    {
        …
    }

I looked a bit at the vim-history repo, and the first tag that has this is Vim 2.4 from 1994, where it's documented as " For MSDOS ignore CTRL-Z at end of file in textmode" but no context on "why?" as far as I can find. I guess it was added for compatibility with older DOS where it was sometimes used, although not needed but "the convention persists in certain circles" (and that was in 2004), and dropping the ^Z was the easiest thing to do.

So the answer is, "you can't really do this in Vim right now", although it's not hard to patch that out (untested, may be incorrect):

if (!error && !got_int && linerest != 0)
{
    …
}

And then use that custom Vim build to edit your MSX files.


I created an issue on the Vim issue tracker to see if this behaviour can be improved: https://github.com/vim/vim/issues/11397

1
  • Great answer! Though the compatibility here is actually with relatively newer disk operating systems such as MS-DOS 1.0 (1981) which stored the exact file length in the directory entry, making it safe to drop the Ctrl-Z. When editing CP/M text files, however, the file's length is always a multiple of the number of bytes in a disk block, so there needs to be an end of file marker, which is Ctrl-Z. Without that, CP/M programs that read the file will read additional garbage characters after the intended file end.
    – cjs
    Oct 19, 2022 at 2:14
0

As we know from Martin Tournoij's excellent answer, the current version of Vim cannot keep the Ctrl-Z EOF marker when editing with fileformat=dos.

A relatively simple workaround (at least on Unix and Windows systems where you have a Bourne shell available) is to execute the following after writing your file:

printf "\x1a" >> filename

Potential improvements to this include writing a key binding or function to execute this from within Vim (using the filename in the "% register), writing a native Vim function that appends the character to the file without having to call an external program, and setting this up to be automatic with a BufReadPost to detect DOS files with the training Ctrl-Z and BufWritePost to add the Ctrl-Z back after writing such files.

There is some code-in-progress in Vim issue #11397 that addresses some of these ideas.

-1

Probably this option can help you:

:set nofixendofline

It prevent Vim for fixing the end of line character of the last line of the buffer.

More information with:

:help 'fixendofline'
1
  • 2
    Sorry, I should have mentioned in my question that I am using nofixendofline already. I've updated the question. nofixendofline does the right thing when editing with fileformat=unix, but doesn't help with the disappearing Ctrl-Z with fileformat=dos.
    – cjs
    Oct 18, 2022 at 18:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.