What is the difference between <C-V>x and \x?

Question

Run the following commands, where <80> is a literal byte entered by typing Ctrl-Vx80:

let literal = "<80>"
let quoted  = "\x80"

• echo(literal) and echo(quoted) will now both output <80>, but,
• len(literal) returns 2 instead of the expected 1, and
• :echo literal[0] outputs <c2>. The expected <80> byte appears to be in literal[1].

Where does this extra byte prefix come from? Why doesn't entering bytes with Ctrl-Vx result in the same sequences of bytes as adding them with a \x special character?

Answer 1

Where does this extra byte prefix come from?

literal holds an UTF-8 encoded string. <80> is encoded in two bytes according to UTF-8 standard specification.

Here bold bits are used by UTF-8 decoder only. The remaining 11 bits are the value of 0x80 prefixed with 3 zero bits.

1 1 0 0 0 0 1 0 | 1 0 0 0 0 0 0 0

• 1 1 0 --- the next char is encoded in two bytes (octets)
• 1 0 --- the "continuation" octet

On the other hand, "\x80" is a "byte" as defined per :h expr-quote, so it never gets encoded in UTF-8. A quote from the help page:

Note that "\xff" is stored as the byte 255, which may be invalid in some encodings. Use "\u00ff" to store character 255 according to the current value of 'encoding'.