2

I am using the most recent nvim in arch.

I have this text showing in my text doc

customerâ<80><99>s 

the <80><99> appears in blue.

The text should read

customer's 

I wish to use this replace format :%s///g to get rid of the â<80><99> and replace it with an apostrophe '

This command :%s/â<80><99>/'/g does not work

How do I do the replacement?

5
  • 3
    See :h i_ctrl-v_digit. However, chances are you open file with wrong encoding.
    – Matt
    Dec 19 '20 at 13:14
  • i did wander about that.
    – Kes
    Dec 19 '20 at 13:26
  • it was pasted in from a PDF doc, so no idea what encoding, if any, I have pasted in there
    – Kes
    Dec 19 '20 at 13:28
  • 2
    The problem is that 0xe2 0x80 0x99 is the "curly" apostrophe in UTF-8 (, rather than '). It seems your Vim things this is ASCII or ISO-8859? Try using :set encoding=utf8 or some such. Dec 19 '20 at 13:37
  • 1
    @martin-tournoij just tried that. It did not work. Sorry to say I copped out and opened the file with Geany and did it there where it was a very simple replacement job. I had to get moving again.
    – Kes
    Dec 19 '20 at 14:21
3

As pointed out, this is an encoding problem. The sequence formed by the three bytes 0xe2 (â), 0x80, 0x99 (three bytes denoted by their hexadecimal number) corresponds to the UTF-8 encoding of Unicode character U+2019, which is the Right Single Quotation Mark:

You can use the Normal-mode command ga to tell you about the character under the cursor (for example,.it will tell you that the "â" has code 0xe2, or that the Unicode character "’" has code 0x2019.)

It may be hard to fix encoding issues, especially when copying and pasting from other applications. PDFs can be a common source of this problem, so I feel your pain there... One thing you could try is to open a buffer in Vim in binary mode with vim -b or :set binary, then save the buffer with the pasted byte sequences, then open the file in Vim again but this time let Vim recognize it as a proper UTF-8 encoded file, hopefully Vim will then be able to interpret the sequences correctly.

To perform search and replace on those expressions, the tricky part is that the <80> and <99> are in fact a single character. You have two options to match them in a search.

First, you can encode them by their character codes. But to do so in a regex, you need to use the \%x sequence, followed by the hexadecimal number representing the character. In your case:

:%s/â\%x80\%x99/'/g

The second option you have is to enter actual <80> and <99> characters in the search expression. One way to do so is to copy them from the buffer and paste them into the search expression.

You can copy them by creating a Visual selection around the â<80><99> sequence and pressing y to copy it into the unnamed register (referred to by the " symbol), then while typing the substitute command, you can use Ctrl+R, " to paste it there.

So what you type is :%s/, then Ctrl+R, " to paste the sequence of three special characters, then type /'/g and then press Enter. What you'll see is:

:%s/â<80><99>/'/g

But the <80> and <99> will be highlighted (in your case, in blue) to indicate they represent a single character each.

2
  • 1
    thank you that is very informative and very helpful :)
    – Kes
    Dec 20 '20 at 17:45
  • 1
    Hoping to save you from Geany next time! 🤣 (Just kidding, always use whatever tool gets the job done. But yeah it's useful to learn how to solve it in the tools you're already familiar with.)
    – filbranden
    Dec 20 '20 at 18:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.