How to show all unique types of a pattern?

Question

I have a very large file containing about, for example, 5 million lines like this:

Songs=****, Singers=****
Singers=***
Songs=***
.... # many many lines
SongType=***
... # many many lines again
Album=***, Singers=**

Since all characters replaced by * are Chinese characters I use /[a-zA-Z]*= to match "Songs", "Singers". And I don't know if I've matched the third: "SongType" since I cannot see it using my eyes, let alone the fourth: "Album".

The problem is that: can I use a vague pattern, in my case [a-zA-Z]*= to know all distinct possibilities(of the same pattern)? I mean all the four strings here.

Of course I can use g command to delete all that matched and see those left. But what if the fifth and others are so rare that I cannot pick them out in the left lines? And in this way the "Album" is deleted and I can never find it.

I have tried to first copy all matched strings to a register and then sort u it in an other file to check, but failed because of the register limitation.

Answer 1

You can do it like this:

:let b:types=[]
:%s/[A-Za-z]*\>/\=add(b:types, submatch(0))/ng
:echo uniq(b:types)

This searches for the pattern /[A-Za-z]*\> and adds the result to the List variable b:types. Since we are only interested in the adding the matches to the list, and don't want to change the buffer, we use the n flag to the :s command. The g flag is there, to get all matches.

Now this will populate your list with all matches, so since we are only interested in the uniq entries, we run the uniq() function on the result again.

Note, the uniq() function is only available with a Vim 7.4.500 or so, so it is relative recent.

If you are further interested in the statistics, you can do it like this:

function! Count(match)
  if has_key(b:types, a:match)
      let b:types[a:match]+=1
  else
      let b:types[a:match]=1
  endif
  return a:match
endfu
let b:types={}
%s/[A-Za-z]*\>/\=Count(submatch(0))/gn
echo b:types

This will count each match. The output of the b:types variable might look like this:

{'SongType': 1, 'Songs': 2, 'Singers': 3, 'Album': 1}