I hate to say it, but it's you. I can relate from personal experience. What seemed to help me is to first set up the gun on the bags pointing at the target. Then insert your RELAXED self behind the rifle. My grip is just a little pressure straight back with whatever part of my fingers are...