2

I have a file exported from some system, and in vi when I open the file, there's a green <92> in it when I open it up where an ' ought to be.

It messed up my bash script and left a line that reads Binary file (standard input) matches after I grep'd it.

When I run file against it, it says the file is encoded as Non-ISO extended-ASCII text, with CRLF line terminators

What is that green <92> and why is it binary and messing up grep?

6
  • 2
    This sounds like an encoding issue. Use the ga command in vim (or the text-filter xxd from your shell) to get a good look at the actual data in the file, then decide what encoding is appropriate, or what happened.
    – D. Ben Knoble
    Dec 19, 2019 at 22:43
  • @D.BenKnoble Is there a way to search for these?
    – leeand00
    Dec 20, 2019 at 12:12
  • @D.BenKnoble Some CTRL+V sort of a thing?
    – leeand00
    Dec 20, 2019 at 12:12
  • Search for what?
    – D. Ben Knoble
    Dec 20, 2019 at 14:03
  • 2
    Yea, use <C-v>u followed by the unicode number that you got from ga
    – D. Ben Knoble
    Dec 20, 2019 at 14:28

1 Answer 1

1

This is a win-1252 quote character which means you're not using the correct encoding. See Unicode characters 2018 and 2019. The encoding is also called CP1252.

In most other encodings, characters between 0x80 and 0x9F are viewed as "graphical controls". Microsoft wanted to add a few more characters to the ISO-8859-1 (probably to have a "better experience" in MS-Word and such where those characters are used a lot). That was a time when writing characters in a word processor or on the screen was not capable of handling a font with more than 256 characters. So having all the characters cramped up in one encoding was useful.

On my end, I often have to change those into code point 0x27 ('), i.e. the apostrophe character.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.