How to combine reading from stdin (- invocation argument) with ++opt

Question

I'm trying to combine two features of vim, but it doesn't seem to be possible.

1: vim -

The first feature is invoking vim with a single-dash argument, which causes vim to read stdin and use the resulting text as the contents of the initial scratch buffer. (Fun fact: vim then uses stderr to attach to the terminal.) For example:

printf 'a\xC0b'| vim -;

The above will cause the initial buffer to have contents aÀb with 'fileencoding' set to latin1.

2: :e++enc=X

The second feature is forcing vim to interpret a file's text in a character encoding different from what vim's autodetection algorithm would settle on. This can be done by running :e ++enc=X where X is the desired encoding. Irrelevantly, I'm also interested in using the ++bad=keep edit option to specify handling of encoding-invalid bytes. So, for example, running :e ++enc=utf-8 ++bad=keep will cause vim to reload the current file assuming its contents comprise UTF-8 text, and keeping bytes that are invalid UTF-8 in the buffer (as opposed to stripping them or replacing them with a valid character, the latter being vim's default).

---

The problem

The problem is that the :e command only works on unmodified non-scratch buffers, since it reloads the entire file from disk. The modification state is not the critical roadblock here, since I could easily :setlocal nomod, but I don't see any way around the scratchness problem. The :e command simply cannot be used in a scratch buffer, since there's no disk file to reload.

Based on my thinking, when loading a file at startup by specifying it as an argument on the shell command-line, I really think there should be some way of communicating to vim how it should interpret the file's text before it loads it the first time, since there's no point in letting vim autodetect an incorrect encoding and then fixing it up after-the-fact by reloading the entire file with an :e command. I think this should apply whether you're reading from stdin or loading one or more disk files at startup, but obviously it's more of a problem for the stdin case, since :e can't work for a scratch buffer, as I described a moment ago.

Here are some things I've tried, knowing they wouldn't work:

printf 'a\xC0b'| vim - ++enc=utf-8 ++bad=keep; ## E492: Not an editor command: +enc=utf-8
printf 'a\xC0b'| vim - -c 'e ++enc=utf-8 ++bad=keep'; ## E37: No write since last change
printf 'a\xC0b'| vim - -c 'e! ++enc=utf-8 ++bad=keep'; ## E32: No file name
printf 'a\xC0b'| vim - -c 'setlocal nomod| e ++enc=utf-8 ++bad=keep'; ## E32: No file name

---

Solution idea

I've thought of one indirect solution, but it's not as convenient as a direct solution would be.

Basically, instead of running the text-generation command in the shell and piping it to vim, I'll run the text-generation command from vim and read its output into a readymade scratch buffer using :r, which fortunately supports the same ++opt interpretation options as :e. Something like this:

:r ++enc=utf-8 ++bad=keep !printf 'a\xC0b';

Vim displays the resulting UTF-8 buffer as a<c0>b.

This works, but it's just not as convenient as it could be. I reiterate my earlier point that there should be some way of specifying the initial encoding interpretation at startup, whether reading from stdin or from a disk file. I'm not aware of such a mechanism, but I'm holding out an ounce of hope that a more knowledgeable vim user will know of a way to do this.

Answer 1

This should solve part of the problem. Try

printf 'a\xC0b'| vim - --cmd "set fencs=utf8" -c "set fenc=utf8"

It reads the file as utf-8 as you wanted, but the ++bad=keep thing is not included.

Answer 2

I've come up with a rather quirky but I think pretty awesome solution to this problem. Upfront warning: It only works from the bash shell on platforms that support process substitution.

The general idea behind the solution is to use a process substitution expansion to read the input from stdin and make it available to vim as a file name. Vim can then read the contents of the temporary file generated by the process substitution mechanism into a buffer using the :read command, which, as I explained in my question, supports the same ++opt options as :edit.

Here's the basic idea:

printf 'a\xC0b'| vim -c 'r ++enc=utf-8 ++bad=keep '<(cat)'| $d' </dev/tty;

As you can see, I am no longer passing the dash argument to vim, and instead I am redirecting its stdin to read directly from the terminal (/dev/stderr would also work here). Hence, instead of having vim read directly from the pipeline, vim initially reads nothing at all; it starts up as normal, creating a single empty scratch buffer during the initialization process. But then, I specify a -c option to execute the :read command with the desired ++opt options, and the file name to read is concatenated into the option value from the process substitution expansion, which of course expands to the temporary file name that contains the output of the background command. In this case, all we need is a simple cat call to get the stdin stream from the pipeline into the temporary file.

I should mention, I actually had no idea that process substitution commands had access to the stdin stream of the enclosing command context, and I was pleasantly surprised to find that they (apparently) do. I don't know (and therefore can't guarantee) that this will work on all systems. The bash manual seems to say nothing about this aspect of process substitution.

Note that the $d addendum to the -c ex command is necessary to delete the extraneous blank line that vim annoyingly adds when you read a file into an empty buffer.

In case anyone's interested, I've enshrined this solution into a more elaborate shell script which automatically parses and relays ++opt options and other vim arguments to a vim invocation constructed within the script. This makes the solution particularly powerful and easy-to-use. Here it is (note: I've named the script vim-):

#!/bin/bash

preVimArgs=();
plusPlusSpecs=();
postVimArgs=();
declare -i i=1;
declare -i gavePlusPlusArg=0; ## assumption
declare -i gaveEdit=0; ## assumption

## start looping through (assumed) pre vim args
for ((i = 1; i <= $#; ++i)); do
    arg="${@:i:1}";
    if [[ "$arg" == ++* ]]; then ## found start of ++opt sequence
        gavePlusPlusArg=1;
        if [[ "$arg" != '++' ]]; then ## guard against treating meaningless "++" as a vim :r modifier; we allow meaningless "++" script args to give the caller a means of separating pre from post even if he doesn't want to pass any "real" ++opt args
            plusPlusSpecs+=("$arg");
            if [[ "$arg" == '++edit' ]]; then gaveEdit=1; fi;
        fi;
        for ((++i; i <= $#; ++i)); do
            arg="${@:i:1}";
            if [[ "$arg" != ++* ]]; then ## found start of post sequence
                postVimArgs+=("$arg");
                for ((++i; i <= $#; ++i)); do
                    arg="${@:i:1}";
                    if [[ "$arg" == ++* ]]; then echo "vim-: error: discontiguous ++opt arguments." >&2; exit 1; fi;
                    postVimArgs+=("$arg");
                done;
                break;
            fi; ## end if (post sequence)
            if [[ "$arg" != '++' ]]; then ## ditto above
                plusPlusSpecs+=("$arg");
                if [[ "$arg" == '++edit' ]]; then gaveEdit=1; fi;
            fi;
        done;
        break;
    fi; ## end if (++opt sequence)
    preVimArgs+=("$arg");
done;
## little post-hoc switcheroo: if no actual ++ or ++opt arguments were given, then move pre args to post, since that's more likely to be the caller's intention when just specifying straight vim args
## if the caller really wants them to be pre, he can specify a meaningless "++" script arg to force all preceding script args to be taken as pre args
if [[ $gavePlusPlusArg -eq 0 && ${#preVimArgs[@]} -gt 0 ]]; then
    postVimArgs=("${preVimArgs[@]}");
    preVimArgs=();
fi;
## also, automatically add ++edit if the caller didn't specify it explicitly; this just makes sense for this script's functionality
if [[ $gaveEdit -eq 0 ]]; then plusPlusSpecs+=('++edit'); fi;

## run the vim command
exec vim "${preVimArgs[@]}" -c "r${plusPlusSpecs[*]} "<(cat)'| $d' "${postVimArgs[@]}" </dev/tty;

Demo:

printf 'a\xC0b'| vim- ++enc=utf-8 ++bad=keep;

Vim displays the resulting UTF-8 buffer as a<c0>b.