There are 1-2 recent discussions on this subforum about this topic and I won't repeat all that data here. The short answer to "does that matter" is NO. But there are details. And it is not just the height that matters but also, shall we call it the "pitch" ... that being, if you have a 20 MOA scope base, then you increase the collimation challenge if you ignore that when mounting the clipon.
.1 inches height difference works for sure on a similar optic for me
.45 inches height difference not working for sure on a similar optic for me
Look around and see if you can find these discussions here. If not, write back here again and I will look them up for you.
And not all clipons are created equal ... the best have collimating lens systems that can mitigate height and pitch mis-match, the others require "adjustments".