9

I was looking at this question and then wondered how I might implement my answer which uses sed using purely POSIX ex.

The trick is that while in sed I can compare the hold space with the pattern space to see if they are exactly equivalent (with G;/^\(.*\)\n\1$/{do something}), I know of no way to do such a test in ex.

I know that in Vim I could Yank the first line and then type :2,$g/<C-r>0/d to almost do what I'm specifying—but if the first line contains anything but very straightforward alphanumeric text this becomes chancy indeed, since the line is being dumped in as a regex, not just a string for comparison. (And if the first line contains a forward slash, the rest of the line will be interpreted as a command!)

So if I want to delete all the lines in myfile that are identical to the first line—but not delete the first line—how could I do that using ex? For that matter, how could I do it using vi?

Is there a POSIX way to delete a line if it exactly matches another line?

Perhaps something like this imaginary syntax:

:2,$g/**lines equal to "0**/d
  • 3
    You could build the command, but it would need a little bit of vimscript and it would probably not be a POSIX way: :execute '2,$g/\V' . escape(getline(1), '\') . '/d' – saginaw Jan 27 '16 at 8:19
  • 1
    @saginaw, thanks. So far the only POSIX approach that has occurred to me is to just use sed as a filter from within ex, and run my entire sed answer on the whole buffer...which would work, of course (and is actually portable unlike sed -i). – Wildcard Jan 27 '16 at 8:32
  • You're right and I find your initial approach with <C-r>0 very good. I'm not sure you could do better with only Ex commands because you have to protect special characters. Without the POSIX compliant constraint I think you would use the very nomagic switch \V and then you would protect the backslash (because it keeps its special meaning even with \V) with the escape() function whose 2nd argument is a string containing all the characters you want to escape/protect. – saginaw Jan 27 '16 at 20:40
  • However, in the previous command I forgot to protect the forward slash too, because it also has a special meaning for the global command, it's the pattern delimiter. So the correct command would probably be something like: :execute '2,$g/\V' . escape(getline(1), '\/') . '/d' Or you could use another character for the pattern delimiter like a semicolon. In this case you wouldn't need to protect a forward slash in the pattern. It would give something like: :execute '2,$g;\V' . escape(getline(1), '\') . ';d' – saginaw Jan 27 '16 at 20:40
  • 1
    I find your second approach with sed also very good. With Vim, you often delegate certain special tasks to other programs, and sed is probably a good example of that. By the way, you don't have to run sed on your whole buffer. If you want to run it only on a portion of the buffer, you can give a range. For example, if you want to filter only the lines between 50 and 100, you could type: :50,100!<your sed command>. – saginaw Jan 27 '16 at 20:53
3

Vim

In Vim you can match any character including newline with \_.. You can use this to construct a pattern that matches a whole line, any amount of stuff, and then that same line:

/\(^.*$\)\_.*\n\1$/

Now you want to delete all lines in a file that match the first, not including the first. The substitution to delete the last line that matches the first is:

:1 s/\(^.*$\)\_.*\zs\n\1$//

You can use :global to make sure that the substitution is repeated enough times to delete all the lines:

:g/^/ 1s/\(^.*$\)\_.*\zs\n\1$//

POSIX ex

@saginaw shows a neater way to do this in Vim in a comment to your question, but we can adapt the above technique for POSIX ex.

To do this in a POSIX-compatible way, you have to disallow multi-line matching, but you can still make use of backreferences. This requires some extra work:

:g/^/ t- | s/^/@@@/ | 1t- | s/^/"/ | j! | s/^"\(.*\)@@@\1$/d/ | d x | @x

Here's the breakdown:

:g/^/                   for each line

t- |                    copy it above

s/^/@@@/ |              prefix it with something unique (@@@)
                        (do a search in the buffer first to make
                        sure it really is unique)

1t- |                   copy the first line above this one

s/^/"/ |                prefix with "

j! |                    join those two lines (no spaces)

s/^"\(.*\)@@@\1$/d/ |   if the part after the " and before the @@@
                        matches the part after the @@@, replace the line
                        with d

d x |                   delete the line into register x

@x                      execute it

So if the current line is a duplicate of line 1, register x will contain d. Executing it will delete the current line. If it isn't a duplicate, it will contain nonsense prefixed with " which when executed is a no-op, since " starts a comment. I don't know if this is the neatest way to accomplish this, it's just the first that came to mind!

It just so happens that the first line can't be deleted because the copying process temporarily changes what line 1 is. If this hadn't been the case you could prefix the :g with a 2,$ range instead.

Tested in Vim and ex-vi version 4.0.

EDIT

And a simpler way, which escapes special characters to create a search pattern (with 'nomagic' set), builds a :global command, then executes it:

:set nomagic
:1t1 | .g/^/ s#\[$^\/]#\\\&#g | s#\.\*#2,$g/^\&$/d# | d x
:@x
:set magic

You can't do this as a one-liner though, since you'd have a nested :global, which isn't allowed.

2

It would appear that the only POSIX way to do this is to use an external filter, such as sed.

For example, to delete the 17th line of your file only if it is exactly identical to the 5th line, and otherwise leave it unchanged, you can do the following:

:1,17!sed '5h;17{G;/^\(.*\)\n\1$/d;s/\n.*$//;}'

(You could run sed on the entire buffer here, or you could run it only on the lines 5-17, but in the first case you're doing unnecessary filtering—no big deal—and in the latter case you would have to use the numbers 1 and 13 in your sed command instead of 5 and 17. Confusing.)

Since sed only does a single forward pass, there's no easy way to do the reverse and delete the 5th line only if it is identical to the 17th line. I tried for a while as a point of curiosity...it's tricky.


Breakthrough - You can do it like so:

:17t 5
:5,5+!sed '1N;/^\(.*\)\n\1$/d;s/\n.*$//'

This is actually the more general method. It can likewise be used to give the same result as the first command (and delete the 17th line only if it's identical to the 5th line) like so:

:5t 17
:17,17+!sed '1N;/^\(.*\)\n\1$/d;s/\n.*$//'

For broader uses such as deleting all the lines of the file that are identical to line 37, while leaving line 37 intact, you could do the following:

:37,$!sed '1{h;n;};G;/^\(.*\)\n\1$/d;s/\n.*$//'
:37t 0
:1,37!sed '1{h;d;};G;/^\(.*\)\n\1$/d;s/\n.*$//'

The conclusion here is, for checking if two lines are identical, the best tool is sed, not ex. But as DevSolar alluded to in a comment, this isn't a failure of vi or ex—they are designed to work with Unix tools; that is a major strength.

  • Much, much harder is: inserting a line at the end of a file, only if the line doesn't already exist somewhere in the file. – Wildcard Jan 30 '16 at 8:05
  • That should be doable with an approach similar to my answer. I don't think it would be a one-liner though! – Antony Jun 21 '16 at 15:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.