Join all non-blank lines

Question

When editing markdown and other text files, I like to use textwidth/wrapmargin to insert line breaks at fixed intervals. (The advantage of this is that each visual line is a Vim "line" - otherwise, "line" and "paragraph" are basically equivalent.) However, when exporting from Vim I want to remove these artificial line breaks, to allow the receiving program to wrap as it sees fit. So I want to transform this

1 Some text that I   |
2 want to transform. |
3                    |
4 And some more text | <-- screen border
5 that I want to     |
6 transform.         |

to this

1 Some text that I want to transform.
2
3 And some more text that I want to transform.

So I tried to join all non-blank lines with

:v/^$/normal J

but that gives me

1 Some text that I want to transform
2
3 And some other text that I want to
4 transform

because it seems to work on every other line, non-recursively.

Is there any way to do this? (If not, I plan to just abandon the fixed textwidth.)

Answer 1

Maybe it sounds too simplistic, but why not

%!fmt -9999

Answer 2

You were almost there:

v/^$/norm vipJ

For each matching line it selects the inner paragraph(without following newline) und joins.This works because, the command is only executed for the first line of each paragraph.

From the vim help about the global command:

The global commands work by first scanning through the lines and

marking each line where a match occurs (for a multi-line pattern, only the

start of the match matters)... In a second scan the [cmd] is executed for each marked line... If a line is deleted its mark disappears.

(Emphasis mine.)

So, when vipJ for the first marked line of the paragraph is executed, all other marked lines for that paragraph disappear, which is also the reason why your solution did not work.

Answer 3

• Normal substitution:

  %s/\([^\n]\)\n\([^\n]\)/\1 \2/
  

• Substitution with magic mode, so as to drop escaping ():

  %s/\v([^\n])\n([^\n])/\1 \2/
  

Explanation:

• ([^\n]) is any character except a newline in the 1st capture group.

• \n is a newline.

• ([^\n]) is any character except a newline in the 2nd capture group.

For each such match, replace it by the 1st captured char, a space, and the 2nd captured char.

Example output. On the left is the original file, on the right is the resulting file.

$ pr -mtw80 -n'|'2 -S'|' file newfile
 1|Some text that I                      |Some text that I want to transform.
 2|want to transform.                    |
 3|                                      |
 4|                                      |
 5|                                      |ABC DEF GHI ZZZ Z1
 6|ABC                                   |
 7|DEF GHI                               |A
 8|ZZZ Z1                                |
 9|                                      |
10|A                                     |B A
11|                                      |
12|                                      |C
13|B                                     |
14|A                                     |
15|                                      |
16|C                                     |

(pr is just formats the output so it is shown side by side, it does not change the files in any way.)

Answer 4

Why it doesn't work

The reason your command works that way is that all the lines are marked for action first, then the join is run (aside: you can use :v/.../j to use the ex-command :join instead of :normal! J). Because the line numbers change, the execution gets off a bit.

Trying :vglobal with a smarter range

One idea is to join from each non-blank line to the next blank line (something like .,/^$/join if the current line is on the start of a paragraph). The problem is that we have to also consider the end of the file—in a range, that would usually be $, but we need to match it in a pattern, so we use \%$.

Trying it out:

vglobal /^$/ .,/^$\|\%$/ join

gives the rather erroneous

Some text that I want to transform.
And some more text that I want to transform.

Fixing a relative range & end-of-file issue

So we've accidentally eliminated the blank line... I thought we could fix that by using .,/pat/-1 join to join from the start to the line above the next blank line, but alas it doesn't work on the sample very well.

So my solution is to double the blank lines and trust the original command to un-double them:

global /^$/ yank | put
vglobal // .,/^$\|\%$/ join

This time I use // with :vglobal because the last pattern is the one we want (and it saves typing). This gives

Some text that I want to transform.

And some more text that I want to transform.

Caveats

This method doesn't work too well when there are multiple blank lines in a row (it doesn't un-duplicate them properly; rather, it removes one from each set). It also doesn't handle the end-of-file already being a single line very well (the range gets confused).

The next section presents a slightly more robust method.

A better solution with pre- and post-processing at the end of the file

Alternately, we can add a blank line to the end of the file, process, and remove it:

$ put =''
vglobal /^$/ .,/^$/-1 join
$ delete

This time we don't even need the complicated pattern for the join range.

Shortening commands

In all cases, the commands I gave can be shortened considerably:

• :vglobal is the same as :v;
• :global the same as :g;
• :join is :j;
• :yank is :y;
• :put is :pu; and
• :delete is :d

Also, I included lots of whitespace around ranges and commands and patterns; you can actually remove every bit of it and it will still work, so the options are

g/^$/y|pu
v//.,/^$\|\%$/j

or

$pu=''
v/^$/.,/^$/-1j
$d

But even most of the seasoned vimmers I know would struggle to decipher these at first glance.