If it’s actually noticeable and repeatable, then one setup works better for you. Which makes sense. Different people have different bodies, likes, dislikes.
In your case, it may be the scope height.
But, that’s not going to be a set rule of thumb. As in, everyone won’t have less wobble with...