4

When editing markdown and other text files, I like to use textwidth/wrapmargin to insert line breaks at fixed intervals. (The advantage of this is that each visual line is a Vim "line" - otherwise, "line" and "paragraph" are basically equivalent.) However, when exporting from Vim I want to remove these artificial line breaks, to allow the receiving program to wrap as it sees fit. So I want to transform this

1 Some text that I   |
2 want to transform. |
3                    |
4 And some more text | <-- screen border
5 that I want to     |
6 transform.         |

to this

1 Some text that I want to transform.
2
3 And some more text that I want to transform.

So I tried to join all non-blank lines with

:v/^$/normal J

but that gives me

1 Some text that I want to transform
2
3 And some other text that I want to
4 transform

because it seems to work on every other line, non-recursively.

Is there any way to do this? (If not, I plan to just abandon the fixed textwidth.)

4

Maybe it sounds too simplistic, but why not

%!fmt -9999
| improve this answer | |
  • The more simple, the better! But I get fmt: invalid width: '9999': Numerical result out of range. fmt -1000 did it. – Quasímodo Jul 27 at 20:11
  • Works perfectly, thanks! For the confused (like me), here is what I think is going on: fmt is the Unix formatting tool that Vim gq uses. This commands runs it with max text width. – Josh Friedlander Jul 27 at 20:17
  • 2
    @JoshFriedlander not quite; vim uses something internally for gq that happens to be a lot like fmt—though you can set the formatprg option to change what gq does! Then :[range]! is an ex-command that filters the range of lines through a given program (as stdin) and replaces them with the result (stdout). – D. Ben Knoble Jul 27 at 21:38
3
  • Normal substitution:

    %s/\([^\n]\)\n\([^\n]\)/\1 \2/
    
  • Substitution with magic mode, so as to drop escaping ():

    %s/\v([^\n])\n([^\n])/\1 \2/
    

Explanation:

  • ([^\n]) is any character except a newline in the 1st capture group.

  • \n is a newline.

  • ([^\n]) is any character except a newline in the 2nd capture group.

For each such match, replace it by the 1st captured char, a space, and the 2nd captured char.

Example output. On the left is the original file, on the right is the resulting file.

$ pr -mtw80 -n'|'2 -S'|' file newfile
 1|Some text that I                      |Some text that I want to transform.
 2|want to transform.                    |
 3|                                      |
 4|                                      |
 5|                                      |ABC DEF GHI ZZZ Z1
 6|ABC                                   |
 7|DEF GHI                               |A
 8|ZZZ Z1                                |
 9|                                      |
10|A                                     |B A
11|                                      |
12|                                      |C
13|B                                     |
14|A                                     |
15|                                      |
16|C                                     |

(pr is just formats the output so it is shown side by side, it does not change the files in any way.)

| improve this answer | |
  • 1
    Works perfectly, +1. This is some black magic... :) thanks for the learning opportunity – Josh Friedlander Jul 27 at 20:18
  • @JoshFriedlander You are welcome, glad it helps! – Quasímodo Jul 27 at 20:22
3

You were almost there:

v/^$/norm vipJ

For each matching line it selects the inner paragraph(without following newline) und joins.This works because, the command is only executed for the first line of each paragraph.

From the vim help about the global command:

The global commands work by first scanning through the lines and

marking each line where a match occurs (for a multi-line pattern, only the

start of the match matters)... In a second scan the [cmd] is executed for each marked line... If a line is deleted its mark disappears.

(Emphasis mine.)

So, when vipJ for the first marked line of the paragraph is executed, all other marked lines for that paragraph disappear, which is also the reason why your solution did not work.

| improve this answer | |
  • 1
    Clever ! Please use :normal! with the bang though – D. Ben Knoble Jul 29 at 15:05
2

Why it doesn't work

The reason your command works that way is that all the lines are marked for action first, then the join is run (aside: you can use :v/.../j to use the ex-command :join instead of :normal! J). Because the line numbers change, the execution gets off a bit.

Trying :vglobal with a smarter range

One idea is to join from each non-blank line to the next blank line (something like .,/^$/join if the current line is on the start of a paragraph). The problem is that we have to also consider the end of the file—in a range, that would usually be $, but we need to match it in a pattern, so we use \%$.

Trying it out:

vglobal /^$/ .,/^$\|\%$/ join

gives the rather erroneous

Some text that I want to transform.
And some more text that I want to transform.

Fixing a relative range & end-of-file issue

So we've accidentally eliminated the blank line... I thought we could fix that by using .,/pat/-1 join to join from the start to the line above the next blank line, but alas it doesn't work on the sample very well.

So my solution is to double the blank lines and trust the original command to un-double them:

global /^$/ yank | put
vglobal // .,/^$\|\%$/ join

This time I use // with :vglobal because the last pattern is the one we want (and it saves typing). This gives

Some text that I want to transform.

And some more text that I want to transform.

Caveats

This method doesn't work too well when there are multiple blank lines in a row (it doesn't un-duplicate them properly; rather, it removes one from each set). It also doesn't handle the end-of-file already being a single line very well (the range gets confused).

The next section presents a slightly more robust method.

A better solution with pre- and post-processing at the end of the file

Alternately, we can add a blank line to the end of the file, process, and remove it:

$ put =''
vglobal /^$/ .,/^$/-1 join
$ delete

This time we don't even need the complicated pattern for the join range.

Shortening commands

In all cases, the commands I gave can be shortened considerably:

  • :vglobal is the same as :v;
  • :global the same as :g;
  • :join is :j;
  • :yank is :y;
  • :put is :pu; and
  • :delete is :d

Also, I included lots of whitespace around ranges and commands and patterns; you can actually remove every bit of it and it will still work, so the options are

g/^$/y|pu
v//.,/^$\|\%$/j

or

$pu=''
v/^$/.,/^$/-1j
$d

But even most of the seasoned vimmers I know would struggle to decipher these at first glance.

| improve this answer | |
  • Not quite working as is. Thanks for the tip about Ex-mode j though! Is vglobal the same as v? – Josh Friedlander Jul 27 at 20:24
  • @JoshFriedlander I've got some working commands, though they are a bit more complex than :%!fmt :) – D. Ben Knoble Jul 27 at 21:39
  • Thanks for the ping, Ben (I remembered the "follow" button exists in the meantime), and also for the explanation. But unfortunately I could not get the 1st option to work on my sample file... I get E16: Invalid range; 8 fewer lines after running the v//.,/^$\|\%$/j command. The 2nd one works correctly. – Quasímodo Jul 28 at 0:09
  • Sure, it is the same one I used in my answer, also here in pastebin: pastebin.com/ndKHyPC0 – Quasímodo Jul 28 at 0:22
  • 1
    Accepted Matt's answer for its simplicity, but I really appreciate the thoroughness and clarity of your answer. Thanks! – Josh Friedlander Jul 28 at 8:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.