2

My issue

I am new to Neovim/Vim, and I am using the Neovim from Scratch config as a starting point. It works well for me for the most part. I have a peculiar encoding issue affecting emoji that prevents emoji from rendering properly from the moment I yank them in Neovim.

Instead of seeing a line like this: "I have a 🎂 (cake) emoji and a 🍏 (apple) emoji"

I see a line that looks like this: "I have a 🎂 (cake) emoji and a 🍏 (apple) emoji"

My question: How do I resolve this emoji rendering issue? Alternatively, where should I look next to troubleshoot this issue?

Steps to reproduce

  • I have text in Neovim that includes an emoji character;
  • My test used this line that I created in TextEdit in plain text mode: "I have a 🎂 (cake) emoji and a 🍏 (apple) emoji";
  • I copied that from TextEdit using the CMD c command, and pasted it into Neovim using CMD v, and it rendered correctly;
  • I then copy the text from Neovim using y in Normal mode, and paste it into TextEdit, and I see "I have a üéÇ (cake) emoji and a üçè (apple) emoji";
  • I did the same test from the command line in iTerm (I've tried the same thing in the Kitty terminal emulator with the same result) where I copied the same time in iTerm2 to TextEdit, and the line pastes correctly;
  • I noticed that if I copy the line from TextEdit using CMD c and then paste into Neovim using p in Normal mode, I see the incorrectly rendered text.

Here is a video demonstrating the issue: https://cln.sh/VPzgydIs5CDYj4cPcXkl

An interesting observation is that if I have the same file open in Neovim and VS Code and update on either side, the other app accurately displays the content with the emoji. This issue only seems to manifest when I use the Vim y and p commands.

My system

  • My encoding in Neovim and iTerm2 is set to UTF-8.
  • I am using macOS 12.1 and Neovim 6.1 (the issue was present in Neovim 6.0 too).
  • I am using the Nerd Font modified version of JetBrains Mono
  • As I mentioned above, I am also using the Neovim from Scratch config for Neovim as I'm a newbie, and this gives me a great foundation to use Neovim.

I'd appreciate any help or pointers here. I could abandon the use of the emoji entirely, although I tend to use them in my daily flow so I'd prefer to be able to keep using them.

8
  • 1
    Is it possible you didn't copy the whole sequence? Or could the app you pasted into not be configured correctly? I just pasted the 🍏 out of your question into vim, yy'd it out, and pasted it into TextEdit, all without trouble. One other place to start: How to debug my vimrc
    – D. Ben Knoble
    Jan 5 at 14:23
  • 1
    Hi @D.BenKnoble - Thank you for your feedback. I just tried that again. I pasted the line with the emoji from the post above into Neovim using CMD v. I then used SHIFT v (so V) to select the line, and then yy to yank it. When I pasted it into TextEdit, I had the same issue. Interestingly when I paste using p in Neovim, the result replaces the emoji with ??. Jan 5 at 14:34
  • 1
    I'll work through the troubleshooting steps. As I mentioned in the video, I also reproduced this in Vim on my laptop. It's possible I have something installed there too, so I'll double check that. If anything, it should be a good control test because my customizations are in Neovim (so .nvimrc and the other config files in ~/.config/nvim/. Jan 5 at 14:38
  • 1
    The debugging steps (commenting half at a time, for example), should apply equally well to neovim as to vim
    – D. Ben Knoble
    Jan 5 at 15:13
  • 1
    I seem to have found the culprit. It is the clipboard = "unnamed" in my options.lua file, here. I have the same issue if I use unnamedplus. I just ran :echo has('clipboard') and received a 1 return. Jan 5 at 16:21

1 Answer 1

1

I believe that I found the solution for my issue. I noticed this issue in the Neovim repo about the !locale options in Neovim. I checked mine and I had the following:

:!locale
[No write since last change]
LANG="en_IL.UTF-8"
LC_COLLATE="C"
LC_CTYPE="UTF-8"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

I'm in Israel, hence the locale. I also noticed a suggestion to add export LANG=C to my shell settings in this AppleExchange discussion so I added that to my .zshrc file. My resulting !locale output is now this:

:!locale
[No write since last change]
LANG="C"
LC_COLLATE="C"
LC_CTYPE="UTF-8"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

This seems to have resolved the issue for me. Copying and pasting with emoji seems to work for me now.

7
  • Results may be even better if you set LC_ALL to the Israel/utf8 locale
    – D. Ben Knoble
    Jan 6 at 12:29
  • Setting LANG="C" looks wrong to me, it's essentially disabling locales... Setting LC_ALL is usually bad, since it overrides every other individual locale choice... It seems to me that the problem is with LC_CTYPE, the value "UTF-8" looks invalid to me, I'm fairly certain that's not a valid choice. I think setting LANG="en_IL.UTF-8" is probably the best way to fix that. (Not sure whether that would fix your problem or not, but that's my best guess here.)
    – filbranden
    Jan 6 at 15:45
  • I tried to set LC_ALL to my actual locale in my .zshrc and received an error when I sourced the updated file to the effect that the declaration wasn't appropriate in that file. 🤔 Jan 6 at 17:55
  • 1
    @filbranden Thanks for your suggestion. Would I declare that in my .zshrc in much the same way? Happy to try that too. I'm just glad I can copy/paste properly now Jan 6 at 17:56
  • Yes, of course, getting the issue solved is worth it! So this discussion is a bit more about perhaps what's the cleanest or purest approach (and possibly one that will avoid problems in other applications.) The Apple StackExchange discussion mentioned preferences under "iTerm: Preferences > Profiles > Terminal > Set locale variables automatically", I wonder if tweaking (disabling?) those would have some effect... They also mention global system preferences for locale. Maybe take a look at those and perhaps post on the Apple SE about how to set them right?
    – filbranden
    Jan 6 at 18:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.