What exactly do patch 8.2.1693 and patch 8.2.1703 do to syntax highlighting?

Question

It is fairly well known that syntax highlighting in vim breaks when one changes colorschemes frequently. This is due to peculiarities of hi def/hi clear.

Recently, the following changes were made:

patch 8.2.1693: "hi def" does not work for cleared highlight
patch 8.2.1703: ":highlight clear" does not restore default link

I would like to know exactly, on a technical level, what has changed after these commits.

Does this fix the problem of changing color schemes fully? What about hi def links in my vimrc? How can I guarantee "colorscheme bar" gives exactly the output that the developer intended?

Answer 1

The actual code of these two commits can be viewed online here and here.

Roughly speaking, Vim now saves value of "def link" in a separate field of the internal "highlight" structure. So upon executing hi clear group(s) are not really cleared, but re-linked to their "def links" (if any).

You can do a quick test like this:

" comments to become 'normal text'
:hi! link vimComment NONE
" restore(?) comments highlight
:hi clear vimComment

This is important, because all (or almost all) color schemes do :hi clear, while hi def link is typically found only inside syntax/*.vim file(s) that are never re-sourced after color scheme change.

So, I believe, the answer is "yes".

What about hi def links in my vimrc?

The first "def link" always wins, so vimrc should get a preference over plugins. Although, one can always do :hi! def link xxx yyy if needed.

Answer 2

Yes, it fixes the issue with colorschemes when they were unable to reset all linked highlight groups to their defaults. Now you can change gruvbox and solarized in the same vim session without "leaked" highlights.

For the :hi def there was a simple fix -- vim considered a cleared highlight as the one that still had settings applied, thus preventing reapply of hi def:

    // Return if "default" was used and the group already has settings.
    if (dodefault && hl_has_settings(idx, TRUE))
        return;

Function hl_has_settings always returned true for the highlight groups that were cleared.

For :highlight clear vim now remembers first default linked group and restores it when needed.

Additional details https://github.com/vim/vim/issues/4405

Me and Antony Scriven are to blame.