Don't want buffer list to be overwritten when I launch vim with filename argument

Question

I have a percent sign in my viminfo string, so my buffer list is saved when I exit vim, and restored when I restart. When I start vim with a filename argument, the buffer list isn't restored, which is what is described in the vim help. However, it's still saved when I exit, which overwrites my longer buffer list. This seems strange behavior. Is there a way to configure vim so I don't lose my saved buffer list when I start vim with a filename argument? The best workaround I can think of is to conditionally add % to my viminfo string in vimrc, when there is no filename argument.