Reliably get the character at a byte index in a string

Question

I'm doing some string processing in vimscript. I obtained a byte index for a character in the string and now want to obtain the character at that byte index. For single-byte charsets (like latin1) this is not a problem:

let char = string[offset]

But what if the string is (potentially) multi-byte?

I implemented the following:

function s:GetCharAtByteIdx(str, index)
  " AFAIK maximum length of utf8-char is 4 byte
  let sp = a:str[a:index:(a:index+3)]
  let chr = strcharpart(sp, 0, 1)
  return [ chr, (a:index + strlen(chr)) ]
endfunction

This function returns the character at the given byte-index and the byte-index of the following character. It works as expected.

Is there a better solution? Maybe a internal function I missed?

---

A (now deleted) comment proposed to use matchstr(str, '\%2c.') to get the character at byte offset 2. The resulting function would be:

function s:GetCharAtByteIndex(str, index)
  let chr = matchstr(a:str, '\%' . (a:index+1) . 'c.')
  return [ chr, (a:index + strlen(chr)) ]
endfunction

The tests were successful, but the test execution time increased from ~10 to ~15 seconds (5 Vim startup, 200 test overall + same for nvim).

Answer 1

Patch 8.2.2233 introduced the charidx() function that returns the character index given the byte index in a string. You can pass this index to the strgetchar() function to get the character at that index.

Answer 2

To answer my own question:

1) There is no internal function to get a character at a byte offset.

2) How is the performance of the script function?

Surprisingly: Very good.

I hacked Vim and added a new function called strcharatbyte(str, index) that returns the same list as the script function s:GetCharAtByteIdx in my question.

Then I ran the tests for my plugin with Vim 50 times in a loop (on Linux).

The average time for the tests with the script function was 4982 ms.

The average time for the tests with the C function was 4468 ms.

The function to get the character at byte offset is called 49334 times during the test. With this calling the C function is approximately 0.01 ms faster than the script function.

This is negligible.

---

The previous version of this answer calculated the advantage of the C with 0.0055 msec. That seemed to good to be true, so I reran the tests.

This time with 50 iterations and also stopped other programs, so the tests are not influenced by other things happening on the computer. The result is still impressive.