5

Run the following commands, where <80> is a literal byte entered by typing Ctrl-Vx80:

let literal = "<80>"
let quoted  = "\x80"
  • echo(literal) and echo(quoted) will now both output <80>, but,
  • len(literal) returns 2 instead of the expected 1, and
  • :echo literal[0] outputs <c2>. The expected <80> byte appears to be in literal[1].

Where does this extra byte prefix come from? Why doesn't entering bytes with Ctrl-Vx result in the same sequences of bytes as adding them with a \x special character?

5

Where does this extra byte prefix come from?

literal holds an UTF-8 encoded string. <80> is encoded in two bytes according to UTF-8 standard specification.

Here bold bits are used by UTF-8 decoder only. The remaining 11 bits are the value of 0x80 prefixed with 3 zero bits.

1 1 0 0 0 0 1 0 | 1 0 0 0 0 0 0 0

  • 1 1 0 --- the next char is encoded in two bytes (octets)
  • 1 0 --- the "continuation" octet

On the other hand, "\x80" is a "byte" as defined per :h expr-quote, so it never gets encoded in UTF-8. A quote from the help page:

Note that "\xff" is stored as the byte 255, which may be invalid in some encodings. Use "\u00ff" to store character 255 according to the current value of 'encoding'.

  • Okay. I think my mistake was in thinking that <80> was a byte too, but if I understand you correctly, it's actually unicode character 0080. `:echo "\u0080" == "<80>" reports 1, so that checks out. Thanks! – Rich Oct 10 at 14:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.