2

I am working on UTF-8 support for my editorconfig-core vimscript plugin. I have hit a strange snag.

This is in official GVim 7.4 Win32. I would like to get this working on 7.4, rather than updating to 8.0, so the plugin can be more widely usable. It has +iconv/dyn, +multi_byte_ime/dyn, and +multi_lang. (Strangely, it does not show +multi_byte, but UTF-8 editing works fine.)

Edit I can also reproduce on Cygwin Vim running in mintty 3.0.0. That is Vim 8.0 with +iconv, +multi_byte, and +multi_lang. Vim encoding is utf-8; $LANG is en_US.UTF-8.

Steps to reproduce

  • Open a new buffer
  • set encoding=utf8<CR>
  • Paste in 中文 (U+4e2d, UTF8 e4 b8 ad; U+6587, UTF8 e6 96 87)
  • Search: /\%u4e2d<CR>. The search should land on the first character.
  • Search using very-magic regexes: /\v\%u4e2d<CR>.
    • When I do this, I get E486: Pattern not found: \v\%u4e2d.

Per this answer, I also tried searching by unicode character entry: /\v<Ctrl+V>u4e2d<CR>. This does return a match.

Why does \%u4e2d not match in very-magic mode, when it does match in magic mode? I also tried nomagic and very-nomagic, and it matches in both of those as well.

Context (aka XY problem)

I am trying to build a regex that will match against a filename. In my particular test case, I have:

regex:    \v\/editorconfig\-core\-vimscript\/localtests\/glob\/\中\文\.txt\_$
test:     /editorconfig-core-vimscript/localtests/glob/中文.txt

Now

:echo match('/editorconfig-core-vimscript/localtests/glob/中文.txt', '\v\/editorconfig\-core\-vimscript\/localtests\/glob\/\中\文\.txt\_$')

returns -1 (no match), even though the similar-seeming match('a','\v\a') returns 0 (match found).

My thought was that I could convert any character >127 into its corresponding \%uNNNN using char2nr() and printf(). However, I ran into the problem above.

Improve this question

1 Answer 1

Reset to default
3

Turns out the "how" answer is very simple: In very-magic mode, % is already magical. Therefore, instead of \%u4e2d, %u4e2d (no backslash) works fine! In full, that's 

/\v%u4e2d<CR>
Improve this answer
4
  • Why did you use \中and not 中? It seems to work as expected without that backslash in recent Vim versions, anyway?
    – Martin Tournoij
    Apr 17, 2019 at 13:50
  • @MartinTournoij It was a side-effect of what I was doing - I was escaping characters for inclusion in a very-magic regex by s/\v([^a-zA-Z0-9_])/\\\1/g, which backslashed everything above char 127. 中 just happened to be the test character that exhibited the problem.
    – cxw
    Apr 18, 2019 at 17:25
  • Right, so removing those backslashes fixes your problem? (they're in the regex in the question, too)
    – Martin Tournoij
    Apr 19, 2019 at 3:31
  • @MartinTournoij Yes, for this and my other recent question, removing backslashes was the answer. This is my first foray into changing magic level, and I am learning as I go.
    – cxw
    Apr 20, 2019 at 0:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.