6

I have a file with a lot of characters represented like this: \u05E2. (These are the actual characters in the file: backslash, lowercase u, and four hexadecimal digits.) Due to that, the file is impossible to read. Is there a way to translate the code to the actual symbol? (in the example above, ע).

10

If I understand you correctly, you have a file where certain characters are represented by their decimal value in the format \uXXXX, with X being any hexadecimal character but always 4, correct?

If so, you can transform those sequences into the actual values they represent by doing a clever search and replace. In this particular case, I would do:

:%s/\\u\(\x\{4\}\)/\=nr2char('0x'.submatch(1),1)/g

This translates to

:%s/      - start a search/replace command in the complete buffer
\\u       - Search for the characters \u
\(...\)   - remember the next chars
\x\{4\}   - 4 hexadecimal characters - they will be remembered and be available as submatch(1)
/         - replace each match by
\=        - evaluate the following to an expression
nr2char(  - return the character for the number given
'0x'      - put a '0x' in front of the number to force hexadecimal value
.         - append
submatch(1) - the hexadecimal number remembered above
)         - closing paren of nr2char()
/g        - replace for every occurrence in each line

While this will replace each occurrence of the pattern \u\x\x\x\x that does not necessarily mean, there will be a glyph available for that character. In such cases it could be shown as a blank square or a '?'.

Note, if you also have less than 4 or more than 4 hexadecimal characters after the \u (or you had an upper 'U' instead of the lower 'u') you would need to adjust the search pattern. This is left as an exercise to the reader.

See the help at :h sub-replace-special and :h nr2char() (among others)

  • Also ensure encoding in Vim is unicode before replacement. – Runium Mar 1 '15 at 16:14
  • That's what the second argument in nr2char() is for – Christian Brabandt Mar 1 '15 at 16:24
  • Yes, but if one have e.g. set enc=latin1 and do the substitution, the u05E2 would result in the glyph â or 0xe2 and not ע. With enc=utf-8 one get ע or 0x05e2. Not sure what exactly happens, but likely nr2char() returns the full utf-8 code, and then it is “cut off” and the leading byte silently thrown away due to enc setting. – Runium Mar 1 '15 at 17:25
  • As the utf-8 sequence for Unicode 05e2 is d7a2 the above assumption is perhaps wrong. – Runium Mar 1 '15 at 17:47
  • @Sukminder - my experience is that unicode encodings are handled specially for situations like this and use the unicode codepoint as the ordinal rather than the multibyte sequence. – Random832 Mar 2 '15 at 19:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.