How to get all distinct words within a set of lines?

Question

I would like to extract a list of distinct words from a set of lines. Is there a way of doing this ?

Say for example I have lines that look like this:

[
[(isPhysicallySettledFxFwd, NO,"Y"),(isPhysicallySettledFxFwd,isPhysicallySettledFxSwap,"N")],
[(isPhysicallySettledFxSwap,NO,"Y"),(isPhysicallySettledFxSwap, isPhysicallySettledCommodity,"Y")],
[(isPhysicallySettledCommodity,NO,"Y"),(isPhysicallySettledCommodity,YES,"Y")]
]

Then i would get a list of distinct words, looking this:

isPhysicallySettledFxFwd
isPhysicallySettledFxSwap
isPhysicallySettledCommodity
NO
YES
Y
N
(
)
"
[
]
,

I am not sure how to even start, apart from copying the lines to Excel and doing lots of manipulations...

Answer 1

You can do something like this:

:let a=[]
:%s/\w\+/\=add(a, submatch(0))/gn
:new
:put =uniq(sort(a))

This will first declare a list a to work with. Then we run a :%s command, to capture all word-characters (\w\+) and act on all matches (g flag of the :s command), but won't actually replace (n flag). We use a sub-replace-expression(\=) in the replacement part, to store the captured submatch in list a.

And finally, we create a new window, and put the unique and sorted (uniq) content of list a into it.

You can get a lot more sophisticated, like only capturing certain words, or counting the numbers, but this shows how flexible the :s command is.

Answer 2

Maybe this:

:%s/\W/\r\0\r/g
:sort u
:g/^\s*$/d

The first puts a line break before and after each non-word character.

The second command sorts the entire file with the option "unique", so all duplicate lines are removed.

The third command deletes all lines that are empty or only contain whitespaces.

Answer 3

You can use grep with the --only-matching/-o flag to accomplish this:

:%!grep -o '\w\+\|\W' | sort -u

Answer 4

If you're open for a plugin solution, my ExtractMatches plugin has (among others) a :YankUniqueMatches command.

:YankUniqueMatches /\w\+/

Result (can be pasted (and :sorted) into a :new buffer with :put):

isPhysicallySettledFxFwd
NO
Y
isPhysicallySettledFxSwap
N
isPhysicallySettledCommodity
YES