Contextual text processing

Question

I would classify my mathematics pdf collection in folders named by the MSC 2010 Classification (just the 2 top-level). My approach is creating a file which will contain all path and uses mkdir in a shell loop.

I clean the data then my file has the following content :

00-XX_General
    00Axx_General_and_miscellaneous_specific_topics
    00Bxx_Conference_proceedings_and_collections_of_papers
01-XX_History_and_biography
    01Axx_History_of_mathematics_and_mathematicians
03-XX_Mathematical_logic_and_foundations
    03Axx_Philosophical_aspects_of_logic_and_foundations
    03Bxx_General_logic
...    ...

How could I transform it to

00-XX_General
00-XX_General/00Axx_General_and_miscellaneous_specific_topics
00-XX_General/00Bxx_Conference_proceedings_and_collections_of_papers
01-XX_History_and_biography
01-XX_History_and_biography/01Axx_History_of_mathematics_and_mathematicians
03-XX_Mathematical_logic_and_foundations
03-XX_Mathematical_logic_and_foundations/03Axx_Philosophical_aspects_of_logic_and_foundations
03-XX_Mathematical_logic_and_foundations/03Bxx_General_logic
...    ...

in vim via command-line mode?

Answer 1

This can be accomplished using only :substitute:

:%s/^\([^ /]\+\).*\n\zs\s\+/\1\/

Here is a breakdown and some intuition on how I created this:

1. :%s/ substitute on all lines. subtlety: this relies on vim processing the substitutes in order, so the initial part of the previous line continues to get captured. I first created this without % and ensure it worked for one line.
2. ^\([^ /]\+\).*\n match anything at the beginning of the line which is not space or /, capturing the characters, anything but newline, then finally a newline. The key here is you can have matches which wrap to the following lines.
3. \zs starting match (i.e., replacement) here. What comes after this will be what we actually want to replace.
4. \s\+/\1\/ any number (at least 1) of spaces, replacing this with what we captured (stored in \1) followed by a /

This can be made a bit shorter by using \v (very magic) and also by changing / to ,:

:%s,\v^([^ /]+).*\n\zs\s+,\1/

Answer 2

I'm not sure this is exactly what you want but you could try this command:

sil exe 'g/^\s\+/s:^\s\+:/:|?^\S?t.-|j!|s/^/ /' | %s/^\s/

Broken down:

               ┌ every time you find a line beginning with a sequence of whitespace
           ┌───┤
           │   │          ┌ put a slash at the end of it
           │   │ ┌────────┤
           │   │ │        │        ┌ then find the previous line which doesn't begin with a whitespace
           │   │ │        │ ┌──────┤ and duplicate it above
           │   │ │        │ │      │
sil exe 'g/^\s\+/s:^\s\+:/:|?^\S?t.-|j!|s/^/ /' | %s/^\s/
                                     ││ │    │    └─────┤
                                     ││ │    │          └ remove a space at the beginning of any line
                                     ││ └────┤
                                     ││      └ add a space in front of the line
                                     └┤
                                      └ join it with the next line

---

Edit:

Peter Rincker has a much shorter solution:

v/^[^ /]\+$/??t-|norm! Js/

Here's how it works:

^[^ /]\+$

This pattern describes a line which doesn't contain any space nor slash (see :h /^, :h /$, :h /[], :h /\+). If you could replace this pattern with the name pat, then you could rewrite the previous command like this:

v/pat/??t-|norm! Js/

This global command tries to find all the lines for which pat is NOT a valid description. See :h :v.

In your example, the following line satisfies this condition:

00Axx_General_and_miscellaneous_specific_topics

For this line, and any other line satisfying the same condition, :v executes this command:

??t-|norm! Js/

The bar is a separation between 2 commands, which are executed consecutively:

??t-
norm! Js/

To understand the 1st command, you must know that all Ex commands may be prefixed by a range, which limits their effect to a certain area of the buffer. Here the Ex command is :t, and ?? is the range. - is just an argument passed to :t.

:t copies/duplicates a line or a group of lines below another arbitrary line. Here, it copies the line specified by ??. This line specifier is described in :h range: it matches the previous line containing the last search pattern. In your case, the last search pattern is:

^[^ /]\+$

So, this command:

??t-

… is the same as this one:

?^[^ /]\+$?t-

And it means: find the previous line which does NOT contain a space or a slash and copy it below the line just above me.

Why just above me? Because - is another kind of line specifier. It's a shorthand for .-1, which stands for the line whose address is equal to the one of the current line (.) minus 1 (-1).

-1 is a numerical offset. And any numerical offset may be used to tweak a valid line specifier. You can express a numerical offset with a sign (- or +) followed by a number, or by piling up - / + signs:

-3    ⇔    ---
+4    ⇔    ++++

In a line specifier, you can also omit the dot, so all of these expressions are equivalent:

.-1    ⇔    .-    ⇔    -

And in the same way, all of these expressions should be equivalent:

.+3    ⇔    .+++    ⇔    +++

So, going back to the line:

00Axx_General_and_miscellaneous_specific_topics

When :v finds it, it executes ??t-. Then :t searches for the previous line which doesn't contain any space nor slash. And :t finds this line:

00-XX_General

Finally :t copies the line just above the current one:

00Axx_General_and_miscellaneous_specific_topics

Which performs the following transformation:

00-XX_General                                           00-XX_General
    00Axx_General_and_miscellaneous_specific_topics  →  00-XX_General
                                                        00Axx_General_and_miscellaneous_specific_topics

This should explain the 1st command, executed by :v every time it finds a line which doesn't contain a space nor a slash.

Now, for the 2nd command:

norm! Js/

This should type the 3 keys Js/ in normal mode. J will join the line which has just been duplicated and the current line, performing this transformation:

00-XX_General                                       00-XX_General
00-XX_General                                    →  00-XX_General 00Axx_General_and_miscellaneous_specific_topics
00Axx_General_and_miscellaneous_specific_topics    

Finally, s/ will delete the character under the cursor and insert a slash. As a result, it will replace the space right after General with a slash.

Note that the whole command will probably not work as expected if one of your lines contains a slash. Also, the bang after :normal is important. Unless you really want the keys to be remapped, always add a bang.

For example, if you use vim-sneak, your s key will probably be remapped to do something entirely different than what it does by default. Usually, you don't want a plugin or a custom mapping to interfere, so it's a good habit to always add a bang after :normal.