Very, very interesting study indeed.
However, I can see a bit of a fundamental problem with the methodology of measuring scopes magnification (which leads to probably incorrect 18x point, which invalidates a number of the study results).
Disclaimer: criticism to follow. Maybe I am wrong all along, and am missing something obvious -- feel free to point it out (I always gratefully recognise my mistakes).
***
I tried to find an explanation to "less than stellar"
optical performance figures for S&B 3-27. With this kind of price tags they better be goddamn perfect, I reasoned, each piece going out of QC, and a gross outright lie as to the max. magnification -- actual 22 instead of announced 27 -- would be a reputational suicide for the company.
Not actually owning a S&B 3-27x, I played with my 3-20x, to see what kind of conclusions one can draw from the method described in the study.
First, a bit of <i>Captain Obviousness</i>:
* The scope's magnification is dependent on the distance from the eye to the eyepiece of the scope. You move your head closer to the eyepiece -- the image gets bigger -- this is your basic human "zoom in" and "zoom out" mechanism.
* Therefore, a "3x zoom" qualification for a scope is only valid for a given eye relief distance. Specs eye relief distances may be different for different scopes.
* "Don't trust the numbers written", yes. For example, the specs of the S&B 3-20x state eye relief of 90 mm. The actual eye relief, where the image completely fills in the eyepiece, at some magnification is closer to 70-80 mm.
* Assuming 90 mm eye relief (that is -- roughly 110 mm to the human eye's focal plane), it is very difficult to respect this constraint with a DSLR camera. The actual focal plane of the camera is inside the camera body, at the back. Add 30 to 40 mm for the actual camera, another (at least!) 30 to 40 mm for the lens; what's left is maybe 40 mm (~1.5 inch) between the camera lens and the scope's eyepiece. At this distance, it appears to me very difficult, if at all possible, to catch at the same time the full image through the scope *and* the non-magnified reference target outside of the scope tube. It is very likely that the DSLR snapshots for the study were taken at a distance well above the recommended eye relief for most scopes.
Conclusion:
Methodologically speaking, in order to really "compare apples to apples", it would have made sense to measure each scope's "perfect" eye relief (e.g. calibrate by the low end of magnification), and make all other measures, such as "true" 18x respecting this (scope's specific) eye relief. In that case, using a compact (non-DSLR) camera would be appropriate, to make sure the camera hardware does not get in the way and limit the minimal distance to the eyepiece.
What actually makes me think that these considerations were not taken into account during the study, is the obvious discrepancy between the "measured" max. magnification for the S&B 3-27x and the field of view figures for the same scope. The FoV is directly proportional to magnification. The ratio between the highest and lowest FoV = 38.7/4.4 = 8.8, which is rather closer to 27/3 = 9 of the announced specs (in reality, it may be a 3-26.4x scope), and does not correspond much to the 22.4/3 = 7.47 ratio that we should have had with the study's 22.4x "measured" figure.
As mentioned in the beginning, this methodological flaw -- failure to account for the scope's specific eye relief -- invalidates many of the study's conclusions (and I would guess it is too late to redo it all).
By the way, the FoV measurements give a much better idea of the zoom range of the tested scopes (rather than relying on the "measured" high end and specs low end). The graph of zoom ranges then looks slightly different from the
study results.
The difference between the blue (claimed) and red (actual) lines pinpoints the "cheaters", where specs don't match reality; really surprising to see the gap for S&B "the King of the Hill" 5-25x -- it does look more like a 7-25x (or 5-17x ?), which many people suspected to be the case, but now, thanks to precisionrifleblog.com, we have figures.
***
Impatiently waiting for the mechanical tests results...
P.S. I wonder if eyebox @ different magnifications has been measured as part of the ergonomics tests; that's another area potentially full of surprises.
