I stand by what I said. The Erector moves downward to increase elevation adjustment, and upward to lower elevation adjustment. When the erector moves downward, as does the image plane, to compensate one must rotate the entire optics platform, IE rifle, upwards in order to compensate for the offset of the adjustment. With the rifle barrel moving upwards indirectly proportionate to the erector system this allows the trajectory of the projectile to reach a higher elevation in regards to the sighting plane. This is the same concept whether you are using iron sights or a rifle scope, and even your base. This is the reason why scope bases are angled DOWNWARD toward the muzzle. This allows the scope to be zeroed closer to the upper limit of travel for the erector system which gives you more downward movement of the erector system, which allows you to reach longer ranges.
This is the best image I could find without drawing my own. The red line represents the erector, or the image plane, and the blue line represents the trajectory of the projectile. If one moves the erector closer, IE downward, to the rifle bore, the trajectory of the bullet rises. If one moves the erector farther from the rifle bore, then the trajectory of the bullet is actually lower.
For this reason I believe the OP has too much 'downward' cant, and therefore if he were to swap his 20 MOA base for a 0 MOA base, he would get rid of his optical distortion at lower magnification at close range.