Use ex-command to check whether two lines are identical?

Question

I was looking at this question and then wondered how I might implement my answer which uses sed using purely POSIX ex.

The trick is that while in sed I can compare the hold space with the pattern space to see if they are exactly equivalent (with G;/^\(.*\)\n\1$/{do something}), I know of no way to do such a test in ex.

I know that in Vim I could Yank the first line and then type :2,$g/<C-r>0/d to almost do what I'm specifying—but if the first line contains anything but very straightforward alphanumeric text this becomes chancy indeed, since the line is being dumped in as a regex, not just a string for comparison. (And if the first line contains a forward slash, the rest of the line will be interpreted as a command!)

So if I want to delete all the lines in myfile that are identical to the first line—but not delete the first line—how could I do that using ex? For that matter, how could I do it using vi?

Is there a POSIX way to delete a line if it exactly matches another line?

Perhaps something like this imaginary syntax:

:2,$g/**lines equal to "0**/d

Answer 1

Vim

In Vim you can match any character including newline with \_.. You can use this to construct a pattern that matches a whole line, any amount of stuff, and then that same line:

/\(^.*$\)\_.*\n\1$/

Now you want to delete all lines in a file that match the first, not including the first. The substitution to delete the last line that matches the first is:

:1 s/\(^.*$\)\_.*\zs\n\1$//

You can use :global to make sure that the substitution is repeated enough times to delete all the lines:

:g/^/ 1s/\(^.*$\)\_.*\zs\n\1$//

POSIX ex

@saginaw shows a neater way to do this in Vim in a comment to your question, but we can adapt the above technique for POSIX ex.

To do this in a POSIX-compatible way, you have to disallow multi-line matching, but you can still make use of backreferences. This requires some extra work:

:g/^/ t- | s/^/@@@/ | 1t- | s/^/"/ | j! | s/^"\(.*\)@@@\1$/d/ | d x | @x

Here's the breakdown:

:g/^/                   for each line

t- |                    copy it above

s/^/@@@/ |              prefix it with something unique (@@@)
                        (do a search in the buffer first to make
                        sure it really is unique)

1t- |                   copy the first line above this one

s/^/"/ |                prefix with "

j! |                    join those two lines (no spaces)

s/^"\(.*\)@@@\1$/d/ |   if the part after the " and before the @@@
                        matches the part after the @@@, replace the line
                        with d

d x |                   delete the line into register x

@x                      execute it

So if the current line is a duplicate of line 1, register x will contain d. Executing it will delete the current line. If it isn't a duplicate, it will contain nonsense prefixed with " which when executed is a no-op, since " starts a comment. I don't know if this is the neatest way to accomplish this, it's just the first that came to mind!

It just so happens that the first line can't be deleted because the copying process temporarily changes what line 1 is. If this hadn't been the case you could prefix the :g with a 2,$ range instead.

Tested in Vim and ex-vi version 4.0.

EDIT

And a simpler way, which escapes special characters to create a search pattern (with 'nomagic' set), builds a :global command, then executes it:

:set nomagic
:1t1 | .g/^/ s#\[$^\/]#\\\&#g | s#\.\*#2,$g/^\&$/d# | d x
:@x
:set magic

You can't do this as a one-liner though, since you'd have a nested :global, which isn't allowed.

Answer 2

It would appear that the only POSIX way to do this is to use an external filter, such as sed.

For example, to delete the 17th line of your file only if it is exactly identical to the 5th line, and otherwise leave it unchanged, you can do the following:

:1,17!sed '5h;17{G;/^\(.*\)\n\1$/d;s/\n.*$//;}'

(You could run sed on the entire buffer here, or you could run it only on the lines 5-17, but in the first case you're doing unnecessary filtering—no big deal—and in the latter case you would have to use the numbers 1 and 13 in your sed command instead of 5 and 17. Confusing.)

Since sed only does a single forward pass, there's no easy way to do the reverse and delete the 5th line only if it is identical to the 17th line. I tried for a while as a point of curiosity...it's tricky.

---

Breakthrough - You can do it like so:

:17t 5
:5,5+!sed '1N;/^\(.*\)\n\1$/d;s/\n.*$//'

This is actually the more general method. It can likewise be used to give the same result as the first command (and delete the 17th line only if it's identical to the 5th line) like so:

:5t 17
:17,17+!sed '1N;/^\(.*\)\n\1$/d;s/\n.*$//'

For broader uses such as deleting all the lines of the file that are identical to line 37, while leaving line 37 intact, you could do the following:

:37,$!sed '1{h;n;};G;/^\(.*\)\n\1$/d;s/\n.*$//'
:37t 0
:1,37!sed '1{h;d;};G;/^\(.*\)\n\1$/d;s/\n.*$//'

The conclusion here is, for checking if two lines are identical, the best tool is sed, not ex. But as DevSolar alluded to in a comment, this isn't a failure of vi or ex—they are designed to work with Unix tools; that is a major strength.