How can I use the `global` command to prepend 'document headers' to 'document lines'?

Question

I am trying to change a text file with data in 'document' form into unnormalized csv. The data are a list of hymn authors ('document header') and for each author a list of one or more hymns they have written ('document line'). The specific operation I am struggling with is taking each 'document header' and prepending it to the one or more 'document lines' that follow it.

Example data

For example, I want to turn this

Beskow Natanael
    140     Ack saliga dag
    399     Tränger i dolda djupen ner
    478     Ditt verk är stort
    479     Kärlek av höjden
Bexell Göran
    103     Herren lever våga tro det
Béze Théodore de
    283     Lovsjung nu alla länder Gud
    360     Såsom hjorten ivrigt längtar

into

Beskow Natanael,140,Ack saliga dag
Beskow Natanael,399,Tränger i dolda djupen ner
Beskow Natanael,478,Ditt verk är stort
Beskow Natanael,479,Kärlek av höjden
Bexell Göran,103,Herren lever våga tro det
Béze Théodore de,283,Lovsjung nu alla länder Gud
Béze Théodore de,360,Såsom hjorten ivrigt längtar

The specific change that I am struggling with is taking the 'document header' and prepending it to each 'document line' so I am leaving the other changes (removing the header, comma between hymn number and hymn title, etc.) to one side for the moment.

Question

I could write a function to do this change, but I find that I am collecting quite a number of ad hoc functions for different editing operations, so I would prefer to learn how to do it with a global command, or, if it is not a suitable operation to perform with global, to understand why. How can I do this operation with global; and, if I can't, why?

What I have tried

I have tried using the :global command to execute different combinations of normal commands.

1) Yank, move, put

g/^\S/normal! y$jPa,

I take this to mean

1. for each line beginning with non-whitespace
2. yank to end-of-line into unnamed register
3. move down one line and put before from unnamed register
4. append a comma

This works, but only for each first 'document line'. The result is

Beskow Natanael
Beskow Natanael,    140     Ack saliga dag
    399     Tränger i dolda djupen ner
    478     Ditt verk är stort
    479     Kärlek av höjden
Bexell Göran
Bexell Göran,    103     Herren lever våga tro det
Béze Théodore de
Béze Théodore de,    283     Lovsjung nu alla länder Gud
    360     Såsom hjorten ivrigt längtar

To repeat the put operation on each 'document line' I think I would need to know how many there are and put in a loop, and I don't see an easy way to do that without writing a custom function.

2) Yank, visually select, substitute

To avoid having to count I thought I could instead visually select the 'document lines' and insert the 'document header' with a substitute over the visual selection.

g/^\S/normal! y$v/^\S^M:s/^\s/\=@" . "," . submatch(0)

I intend this command to mean the following

1. for each line beginning with non-whitespace
2. yank to the end of line
3. visually select until the next line beginning with non-whitespace
4. substitute over visual selection
   • match beginning-of-line + whitespace
   • substitute contents of unnamed register + a comma + the entire match (i.e., the whitespace)

This command does not work at all.

• It does not change the buffer at all.
• It leaves me in visual mode, with the last character on the first line selected.
• Hitting /<Up> to look at the last search pattern shows /^\S^M:s/^\s/\=@" . "," . submatch(0). This tells me that the ^M does not execute the search as I had hoped, but rather everything following / is interpreted as part of the pattern.
• Checking the unnamed register after running the command, it contains the last 'document header' in the buffer. This tells me that it goes through the buffer and performs the yank operation.

If, however, I type out the normal mode commands manually they work as expected.

Answer 1

This just screams "macro and normal" for me. First, a comma after each name would be helpful as a name terminator, so execute :g/^\S/normal A, first.

Start somewhere on line 2 and type

• qq to start recording into buffer q
• 0ma to go to the start of the line and save the position at mark a
• ?^\S<Enter> to search backward for the next line that does not start with a whitespace (you could also just go 1 line backwards, but add a 0 in that case to go to the start of the line)
• yf, to yank to the first comma, including it, which should be the name by what we did in the first step
• 'a0 to return to the line of the mark a and go to the beginning (use `a instead, we saved at the beginning of the line, but I can't type that here with the grey background)
• p to paste
• 0x to delete the whitespace at the beginning
• q to stop recording

That seems to do what you want, adjust accordingly (you could skip the first step in various circumstances, e.g. if all the lines that start with a whitespace have a numeral as the first non-whitespace character... but since you wanted to add commata anyways, I went that way). Now just use :g/^\s/normal! @q to execute the macro we saved in buffer q (@q) on each line that starts with a whitespace (^\s).

This works for your example. It has the additional advantage of being adjustable for the other changes you probably want to do, just by changing the macro when recording it.

See

• http://vim.wikia.com/wiki/Macros
• https://stackoverflow.com/questions/390174/in-vim-how-do-i-apply-a-macro-to-a-set-of-lines

for more.

I noticed you might want to get right of the author-only lines, and that might be hard later on. Just insert a :g/^\s/normal J after the first step, which produces 1 correct line and gets rid of the author-only line for each author, and then proceed with the macro as given.

Answer 2

This is one way to do it (I'm sure it's not perfect, but it gets the job done):

:g/./if getline('.')[0] =~ '\w' | delete | else | s/\s\+\(\w\+\)\s\+\(\w\+\)/\=@" . ',' . submatch(1) . ',' . submatch(2)/ | -join! |endif
:g/^$/delete

BTW, this would look much nicer if you would allow yourself to use functions.

The idea is, you use global to match both headers and non-headers (=> non-empty lines), but act differently on them: if it's a header, delete it (=> it's yanked to the default register), if not, replace first spaces with the current header (by yanking it back from the default register) followed by a comma, and the next spaces with a comma.

If, instead, you only search for headers, it's harder to then act on non-header lines.

Answer 3

The second command is missing a final ^M.

In order to understand why the second command works if the normal mode commands are executed manually, but not from within the global command, I recorded the successful sequence of commands as a macro. In the register I find

y$v/^\S^M:s/^\s/\=@".",".submatch(0)^M

Lo and behold, if I use this sequence with global, which gives an identical command to the second one in the question except for the additional ^M at the end

g/^\S/normal! y$v/^\S^M:s/^\s/\=@".",".submatch(0)^M

it works.

It does not, however, finish the last 'document' in the buffer, since there is no start-of-line+non-whitespace to search forward to.

I'm a little disappointed about that being the solution and I'm not sure whether I should edit the question, close it, or leave it like it is. I'll leave it for a while and see if there are more answers, explanations or comments–I'm sure that, even if this global command now works (except for the last line), it could still be improved upon.