What are the rules for "w" word motion when characters are not in iskeyword?

Question

If I have a shell script with code like

...
chars+=('5')
chars+=('6')
...
chars+=('%')
chars+=('&')
...

and iskeyword is defined as

iskeyword=@,48-57,_,192-255,-

This definition indicates to me Vim will consider a continuous sequence of characters from these iskeywords characters to be a word.
So if I am here

chars+=('5')
 ^ (cursor)
chars+=('6') 

and press w

chars+=('5')
     ^ (cursor)
chars+=('6')

I end up on + which is not the beginning of a word as defined by iskeyword character set. Why? Does Vim stop at the beginning of a group of these non-iskeyword chars because it considers you might be interested in doing something with them?

In this case, wouldn't it be better to say w takes you both to the beginning of the next "word" OR the beginning of the next "anti-word" (continuous sequence of characters NOT defined in iskeyword)?

Because code contains a lot of characters not in iskeyword, I am interested in knowing the rules for "w" word motion regarding these non-word characters.

Answer 1

I think the misunderstanding is that iskeyword doesn't define what a word is; rather, it adjusts where a boundary may happen by adjusting what a keyword is. Both keywords and non-keywords are still "words".

According to :help word:

A word consists of a sequence of letters, digits and underscores, or a sequence of other non-blank characters, separated with white space (spaces, tabs, ). This can be changed with the 'iskeyword' option. An empty line is also considered to be a word.

But, that doesn't quite answer your question exactly.

My understanding is that "iskeyword" is used to group the containing characters as a type of word called "keyword", but that doesn't mean that characters (except whitespace) outside of iskeyword aren't words, but are just non-keywords. Also, it seems that letters and high unicode is considered part of a keyword regardless (from help 'iskeyword' and help 'isfname').

As such, when you press w at the beginning of chars+=, the letters are considered part of the same word because they form a keyword, but since + and = are not in iskeyword, that makes them a non-keyword word, which means there is a word separation.

The behavior that I observe is that, outside of whitespace separation, keywords and non-keywords are considered words, and a transition from keyword to non-keyword or from non-keyword to keyword forms a word boundary, just like whitespace forms a word or WORD boundary.

Answer 2

I get more confused after reading @John O'M 's answer. But after some search, I get his point.

Here is my notes:

Character classes:

\i      identifier character (see 'isident' option)     /\i
\I      like "\i", but excluding digits                 /\I
\k      keyword character (see 'iskeyword' option)      /\k
\K      like "\k", but excluding digits                 /\K

\w      word character:                 [0-9A-Za-z_]    /\w
\W      non-word character:             [^0-9A-Za-z_]   /\W   (ps: not limited to ASCII, can match Chinese charaters)

Modified from nvim's help

 word/WORD
    I. consist of a sequence of
          group  1.  letters, digits and underscores,  【the definition of `word` can be changed with the 'iskeyword' option. 
                                                           After `set iskeyword+=-`,   `-` becomes a member of group 1
          group  2.  other non-blank characters, (including non-word characters, e.g. Chinese charaters)
                            
                             (when characters in group 1 are seperated by characters in group 2, 
                             such as "I爱you", character "爱" makes the `WORD` become three `word`s.  
                             Similarly, the `WORD` "我n你" contains 3 `word`s
                               
    II. separated with:
                       white space  (spaces,  tabs, <EOL>).

I like the use of "XOR" in @Nicholas Cousar's comment:

(modified by me)

A word consists of a sequence of
group 1 (letters OR digits OR underscores i.e. keywords),
XOR
group 2 (a sequence of other non-blank characters i.e. non-keywords),
separated with white space (spaces, tabs, )

Real Example

https://github.com/dominikduda/vim_current_word

pain point: it solves all the problems other similar plugins have with knowing what exactly is a word and when cursor is on the word.

Plugin determines what is a word based on what is a keyword in current file type. For example foo#bar in .vim file will be a single word (because vimscript allows to put # as part of variable name), but in .rb file foo#bar will be considered as two words (because # starts comment).

Helps predict how vim word movements (w, e, b) will behave

Plays well with * and #

However, this plugin does not work so well as described !!

This plugin will highlight I爱U as a word, I-U will not be highlighted as a word ,because - is not in my iskeyword (iskeyword=@,48-57,_) . But nvim' s word motions, e.g., w , *, and # regard I爱U as 3 words.