The targets.vim plug-in implements something very similar to what you described, the main difference being that it has two separate text objects one for "any block" (ib
, etc.) which works for ()
, []
and {}
and another one for "any quote" (iq
, etc.) which works for single, double or back quotes.
See the README section on Multi Text Objects for more details.
The main reason for separate objects for blocks and quotes is that blocks typically nest, while quotes do not.
In any case, the very existence of this plug-in that implements a text object which will match one of several delimiters based on context should demonstrate that it's definitely possible to implement a smart text object. If you really want one that matches either blocks and quotes, you might be able to draw on the ideas of this plug-in in implementing your operator.