0

I would like to join all lines in a file which are continuous parts of some natural language but not back-to-back lines which are two separate entities. For example:

Join:

There was once a man
called Bob and he lived
in France.

Do not join:

Chapter 1
The arrival
Part 1

I am starting to think this will only be convenient with a modern machine learning technique as otherwise there will not be enough distinguishing features in the text to make a rule-based program. The script basically has to know on a conceptual level what is a self-sufficient unit of text (i.e. a header) vs. what seems like broken natural language, including maybe that it connects to the previous or following text.

If I can train some machine learning function to detect whether or not a line should be joined with the next or previous, either a Boolean true or false or returning “join next”, “join previous” or “do not join”, how could I implement it in Vim? A function which goes through every line, calls the classifier, and calls the Vim join method if it passes? Or is there a better way?

1 Answer 1

1

Imagine you have a program (naturaljoin) that does that analysis and produce the result you need.

Then you can filter buffer contents using it:

:%!naturaljoin

Help topic :h filter might be worth reading. Also :h 'formatprg'.

PS, code formatters are very often used with this in mind, e.g. :%!gofmt

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.