3

I'm doing some string processing in vimscript. I obtained a byte index for a character in the string and now want to obtain the character at that byte index. For single-byte charsets (like latin1) this is not a problem:

let char = string[offset]

But what if the string is (potentially) multi-byte?

I implemented the following:

function s:GetCharAtByteIdx(str, index)
  " AFAIK maximum length of utf8-char is 4 byte
  let sp = a:str[a:index:(a:index+3)]
  let chr = strcharpart(sp, 0, 1)
  return [ chr, (a:index + strlen(chr)) ]
endfunction

This function returns the character at the given byte-index and the byte-index of the following character. It works as expected.

Is there a better solution? Maybe a internal function I missed?


A (now deleted) comment proposed to use matchstr(str, '\%2c.') to get the character at byte offset 2. The resulting function would be:

function s:GetCharAtByteIndex(str, index)
  let chr = matchstr(a:str, '\%' . (a:index+1) . 'c.')
  return [ chr, (a:index + strlen(chr)) ]
endfunction

The tests were successful, but the test execution time increased from ~10 to ~15 seconds (5 Vim startup, 200 test overall + same for nvim).

  • imo, your first solution is totally fine and efficient. Was there a reason you didn't use a:str[a:index:(a:index+3)] to get the first four bytes? – Mass Apr 22 at 18:24
  • I usually split() the string and return the index. – Christian Brabandt Apr 22 at 18:52
  • @mass Thanks, fixed (here and in my plugin). – Ralf Apr 22 at 20:39
  • @ChristianBrabandt How would split() help with the byte offset? – Ralf Apr 24 at 16:41
  • @Ralf Ah you have a byte offset, I missed that. Where does that come from? Is this from compiler/linting warnings? – Christian Brabandt Apr 25 at 5:45
2

To answer my own question:

1) There is no internal function to get a character at a byte offset.

2) How is the performance of the script function?

Surprisingly: Very good.

I hacked Vim and added a new function called strcharatbyte(str, index) that returns the same list as the script function s:GetCharAtByteIdx in my question.

Then I ran the tests for my plugin with Vim 50 times in a loop (on Linux).

The average time for the tests with the script function was 4982 ms.

The average time for the tests with the C function was 4468 ms.

The function to get the character at byte offset is called 49334 times during the test. With this calling the C function is approximately 0.01 ms faster than the script function.

This is negligible.


The previous version of this answer calculated the advantage of the C with 0.0055 msec. That seemed to good to be true, so I reran the tests.

This time with 50 iterations and also stopped other programs, so the tests are not influenced by other things happening on the computer. The result is still impressive.

  • I accepted my own answer, but feel free to comment, criticize or provide a different answer. – Ralf Apr 24 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.