It didn't used to do this. I'm not bumping the scroll wheel.
There's a setting in Photoshop zoom control menu that will fix this. just go to Settings>Options> Edit>Zoom Controls>Turn off dumb stuff
You can set the scroll wheel actions in Edit, preferences