why does cpo-=c cause perceptual “lag?”

Question

I'm using a CursorMoved autocommand with searchpos to do highlighting of special words. I've been using set cpo-=c to handle possible overlapping matches, and I restore it afterwards. This set seems to be causing a lot of cursor lag when scrolling with hjkl.

However, I don't understand where the lag is coming from. I wrapped the set in a reltime:

let a = reltime()
noautocmd set cpo-=c
let g:foo = reltimestr(reltime(a))

I inspect g:foo and it's 0.000025; much, much too low to be causing any sort of lag. In fact, my entire CursorMoved autocommand takes no more than 10 milliseconds. Replacing the set with a sleep 10m removes the "lag" entirely.

Another odd thing is the lag happens when using vim over SSH, but not locally. So, why does set cpo-=c appear slow, but is not actually empirically slow?