Define a match group for a modeline

Question

Looking to define a match group for modelines. vim defines two patterns for modelines:

The first form:

[text{white}]{vi:|vim:|ex:}[white]{options}

• [text{white}] empty or any text followed by at least one blank character (<Space> or <Tab>); ex: always requires at least one blank character
• {vi:|vim:|ex:} the string vi:, vim: or ex:
• [white] optional white space
• {options}a list of option settings, separated with white space or :, where each part between : is the argument for a :set command (can be empty)

Examples:

   vi:noai:sw=3 ts=6 ~
   vim: tw=77 ~

The second form (this is compatible with some versions of Vi):

[text{white}]{vi:|vim:|Vim:|ex:}[white]se[t] {options}:[text]

• [text{white}] empty or any text followed by at least one blank character (<Space> or <Tab>); ex: always requires at least one blank character
• {vi:|vim:|Vim:|ex:} the string vi:, vim:, Vim: or ex:
• [white] optional white space
• se[t] the string set or se (note the space); When Vim is used it must be set .
• {options} a list of options, separated with white space, which is the argument for a :set command
• : a colon
• [text] any text or empty

Examples:

   /* vim: set ai tw=75: */ ~
   /* Vim: set ai tw=75: */ ~

I currently have a simple pattern to match the modelines I've written:

syntax match Modeline ".*\s+vim:\sset.*:.*"

but I would like to create a match group that matches on all lines Vim treats as modelines (the pattern above misses some of the nuance with ex for example). I imagine vim internally has regex expressions that match these patterns. I'd like to use those expressions in a match group.

Answer 1

Alternative contortions based on :help modeline that match
the same (13) lines in :help options.txt as does the other
regexp, and are also accommodating for \sex: and no
leading text.

(Note that it is assumed that {vi,{V,v}im,ex}: is within
or after the leftmost 16 (arbitrary) characters and some
blanks, and we only seek set\s for Vim: because set\=\s
can be optional for {vi,vim,ex}:.)

syn match Modeline /\%#=1^\C\%(.\{-,16}\s\+\)\=\%(\%(vim\=\|\s\@1<=ex\):\|Vim:\s*set\s\).*$/

It may be more readable as a pair of definitions:

syn match Modeline /\%#=1^\C\%(.\{-,16}\s\+\)\=\%(vim\=:\|Vim:\s*set\s\).*$/
syn match Modeline /\%#=1^\C\%(.\{-,16}\s*\)\=\sex:.*$/

The following ‘core dump’ should also cover versioning
(:help modeline-version).

syn match Modeline /\%#=1^\C\%(.\{-,16}\s\+\)\=\%(\%(vim\=\%(m\@1<=[<=>]\=\d\+\)\=\|\s\@1<=ex\):\|Vim\%([<=>]\=\d\+\)\=:\s*set\s\).*$/

And still a ‘core dump’ :( as a triplet:

syn match Modeline /\%#=1^\C\%(.\{-,16}\s\+\)\=vim\=\%(m\@1<=[<=>]\=\d\+\)\=:.*$/
syn match Modeline /\%#=1^\C\%(.\{-,16}\s\+\)\=Vim\%([<=>]\=\d\+\)\=:\s*set\s.*$/
syn match Modeline /\%#=1^\C\%(.\{-,16}\s*\)\=\sex:.*$/

(The key to dense look-behinds \@1<=:

• If a string without a trailing blank is matched with
  \%(.\{-,16}\s\+\)\=, fail ex: with \s\@1<=ex;
• If vi: is matched with vim\=:, fail for versioning with
  m\@1<=[<=>]\=\d\+.)

---

I did a bit of profiling with :help :syntime in v9.1 (with
1-16 patches). It turns out that the linked pattern can be
made about four times nimbler by rewriting it as follows:

syn match Modeline /\%#=1^.*\s\%(\%(vim\=\|ex\):\|Vim:\s*set\s\).*$/

In order to recreate the profiling setup I have used:

1. Write this syntax file (~/.vim/syntax/modeline.vim),
   and change PATTERN:

syn match Modeline /PATTERN/
hi def link Modeline Todo

2. Write this definition of g:Auto_Scroll_And_Report() in
   /tmp/report.vim:

vim9script

def g:Auto_Scroll_And_Report()
    cursor(1, 1)
    const last: number = line('$')
    const walk: string = 'normal! ' .. "\<C-D>"
    &l:scroll = winheight(0) / 2
    syntime on

    while line('w$') != last
        execute walk
        redraw
        sleep 25m
    endwhile

    syntime report
enddef

3. Rename a copy of $VIMRUNTIME/doc/options.txt as
   /tmp/options.modeline and change &ft for its modeline:

 vim:tw=78:ts=8:noet:ft=modeline:norl:

4. Repeatedly launch and profile Vim:

vim --clean /tmp/options.modeline -Nu NONE \
  -c 'set rtp+=~/.vim' \
  -c 'setf modeline' \
  -c 'syntax on' \
  -S /tmp/report.vim \
  -c 'call Auto_Scroll_And_Report()'

WHEN YOU MODIFY SYNTAX PATTERNS, MAKE SURE THEY STILL MATCH
THE EXPECTED ITEMS BEFORE YOU PROFILE THEM AGAIN.

Answer 2

(Not an answer proper, but swaths of code don't belong to
comments either.)

Take a look at chk_modeline() in src/buffer.c and
src/testdir/test_modeline.vim.

Consider cloning vim/vim and vim/vim-history and tracing
code evolution as follows:

cd path/to/vim/vim/src/
git log --oneline -- testdir/test_modeline.vim

git log -L '/^chk_modeline(/,/^}$/':buffer.c HEAD

cd path/to/vim/vim-history/src/
git log -L '/^chk_modeline(/,/^}$/':buffer.c HEAD

git tag --list v[34]\*
git switch --detach v3.0
git log -L '/^chk_mline(/,/^}$/':fileio.c HEAD

git tag --list v1\*
git switch --detach v1.14

Now, you can start with a regexp prototype for v1.14 and
reiterate it as far as you need it.

Answer 3

I followed @Friedrich's suggestion and built a pattern that matches many of the common modelines:

syntax match Modeline /.*\s\+\(vi\|vim\|ex\):.*\|.*\s\+\(vi\|vim\|Vim\|ex\):\s*set.*:.*/

it's missing the mandatory space before ex: but is working well in practice for me.

Edit: following some of the suggestions from below.