I have observed the same thing as bruddah...different bullets give different dimension to just touching when using the Hornady OAL gauge and calipers with the Hornady insert. One thing you'll notice if you compare the Hornady caliper insert with other brands like Sinclair is that the hole for a given caliber is actually smaller. Therefore it seats farther out on the ogive than do the inserts with holes closer in diameter to the actual caliber. There are both good points and bad that go along with using the Hornady inserts, but overall I generally prefer them. If you ever have to seat a bullet fairly deep in the case and/or trim your necks on the long side, the Sinclair insert seats very close to the case neck and can make it a little more challenging to get good readings.
To answer your question, when measuring your chamber with the Hornady OAL gauge (Stoney Point gauge), the nature of the tool means you have to rely on "feel" as the bullet approaches/touches the lands. Doing this consistently takes a bit of practice, but I don't find it very difficult. Because you are pushing the bullet to where it just touches the rifling, it is possible that there may be a difference in how you perceive the "touch" because of the different radii of curvature for the ogives of different bullets and how they behave when they encounter the initial angle of the lands in the throat.
However, it is much more likely that because the Hornady insert has a smaller hole and actually contacts the bullet much farther away from the bearing surface/ogive junction, you are simply contacting the different bullets at different points on the ogivewhen you measure, because their ogive radii are not the same. In other words, the diameter of the ogive at which different bullets contact the rifling is the same; obviously the bore/rifling diameter doesn't change just because you switched bullets. Because the Hornady comparator insert isn't actually contacting that point, but a point that matches the smaller diameter of the insert hole that is farther out on the bullet ogive, bullets with different ogive radii will change where the comparator insert seats on the bullet when you measure. The larger the difference in ogive radii between the bullets, the larger the distance between where the comparator insert will seat. Hence, the larger the discrepancy you will find when using this system to measure your chamber with different bullets.
The best way around this is to use a comparator insert that seats as close to the ogive/bearing surface as possible (ie. a larger hole). The Hornady insert is obviously not the best tool for this task because it seats farther out on the ogive. The closer the comparator insert seats to the bearing surface, the less discrepancy you will see between different bullets of the same caliber. Bottom line is that if you are using the Hornady OAL gauge in conjunction with Hornady comparator inserts, you will need to re-measure your chamber dimensions when switching to a different bullet. I don't find it to be that big of a deal, but it does mean a little more work when trying different bullets. Alternatively, just buy a comparator insert that seats closer to the bearing surface like the one below. They're not very expensive.
SINCLAIR INSERT STYLE BULLET COMPARATOR | Sinclair Intl Edited to add: here are chamber measurements (CBTO, just touching the lands) I took from one of my rifles using the Hornady OAL gauge and calipers with the Hornady comparator insert (avg of 10, +/- SD, numbers not rounded off)
Berger 185 Juggernaut: 2.2130" +/- 0.0
Berger 168 Hybrid: 2.1927" +/- 0.0003"
Note that the difference between the two is approximately 20 thousands. As bruddah mentioned, that could make a big difference in pressure between loaded rounds depending on how far off the lands you typically start your charge weight testing. You're much better off (safer) either measuring the chamber separtely for each different bullet you use, or buying a comparator insert that seats closer to the bearing surface as mentioned above. Good luck.