4

I have noticed that along with most .spl files, there is also a .sug file. As far as I could gather from my search with :help, this is a file which helps Vim rank suggestions from the .spl file.

I have tried to search the internet for an explanation of how these .sug files work, but with no luck. I have a suspicion, that they have something to do with the numbers that appear if your use :spelldump! instead of :spelldump to dump a word list.

Can anybody tell me how they help rank the suggested words, if this is indeed what they do. And possibly how one could generate a custom .sug file if this makes sense to have.

When I searched the internet most hits I got involved Vim. Are .sug files a Vim concept?

5

The first hint as to the purpose of these files is :h spell-sug-file:

This file is used to find suggestions by their sound-a-like form quickly. At the cost of a lot of memory

There are three types of files used in Vim's built-in spell checking tool:

Dictionary Simple lists of correct/known words that are used to identify spelling errors. Vim scans all words and tries to find each in the configured dictionaries. If a word is not found it can be highlighted as a spelling error.

SPL file These files are used to look up suggested corrections for common mistakes such as "purposeful" misspellings (e.g. suprise for surprise) and typos (e.g. teh for the). The files contain trie data structures in binary format.

SUG file These files are used to look up suggested corrections for words that are "sound-alike" or phonetically similar (e.g. funetik for phonetic). As noted in the side bar below these suggestions are primarily useful for people who don't know the language well. Like SPL files, these files contain trie data structures in binary format. The key values are "soundfolds" which are a kind of normalization of phonetic spellings. A misspelled word is converted with the soundfold() function and the result is used to lookup suggestions in the SUG data. There's more information about soundfold/sound-alike conversion and lookup in this Phonetic Code document.

I sussed out most of this from the spell.txt Vim help file but there are also some interesting tidbits in the Vim developer notes at h: develop-spell and h: develop-spell-suggestions

Some more points of interest:

  • For some languages, English being a notable exception, these .sug lists can be much larger than their equivalent .spl list. Spanish and Russian are two examples. This is one reason for the warning above about memory usage.
  • The 'sug' file extension is a Vim convention (possibly copied from somewhere else.) The concept of matching phonetic variations is not a Vim innovation.
  • If .spl and .sug files for a language are not installed Vim will try to download them from the Vim ftp server using the Netrw plugin. (English versions are installed by default.) I don't think you'd want to try to generate these yourself.
  • To make use of the phonetic lists you need to set 'spellsuggest' to best or double.

As for scoring, short of looking at code the best I can do is copy in h: spell-double-scoring:

The 'spellsuggest' option can be used to select "double" scoring. This mechanism is based on the principle that there are two kinds of spelling mistakes:

  1. You know how to spell the word, but mistype something. This results in a small editing distance (character swapped/omitted/inserted) and possibly a word that sounds completely different.

  2. You don't know how to spell the word and type something that sounds right. The edit distance can be big but the word is similar after sound-folding.

Since scores for these two mistakes will be very different we use a list for each and mix them.


Side Bar: An interesting aspect of the phonetic lists is that they are mostly useful for non-native speakers of a language since native speakers will not often make mistakes that are phonetically similar while at the same time not the type of error that the normal suggestions identify. On the other hand, non-native speakers that have no idea at all about a word's spelling will often try to "sound it out" and use that spelling. For example, "funetik" in place of "phonetic". A normal suggestion list is unlikely to get a match on that. :h spell-double-scoring briefly mentions this differentiation.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.