Presuming you are only interested in replacing the numbers before the first ampersand on each line, one fairly crude way to do it is to first split the lines on the first ampersand, then perform the substitutions, and then rejoin the lines:
In the first line, the
^M is a literal carriage return which you can enter by pressing: Ctrl-VEnter. A more copy-and-pastable version of that line is as follows:
:execute "g/\&/norm! f&i\<CR>"
If, on the other hand, you want to replace any integers that come before any ampersand, or if each line is guaranteed only to have one ampersand, then a one-liner is possible: