Once mirage and/or lighting starts to muddy things up between you and the target, there isn't much you can do as far as optics on the firing line to be able to see individual shot holes reliably.
To some degree, better optics *do* help - a high end spotting scope will do better (usually) than some cheap POS you bought at the local sporting goods store (or off Amazon/Ebay/Alibaba). But at some point, once the shot holes get close enough together, or close to / into a line on the target.... you're SOL as far as using glass.
As mentioned above, target cameras can help - though they sometimes have their own foibles. Electronic target systems (i.e. ShotMarker) sometimes cover a large target better, but have their own quirks as well. The question is, will the host range let you put something like that up downrange? A lot of times, the answer is 'No', particularly during competitions.
Before I had a ShotMarker, when I would go to set up for load testing @ 600yds, I'd put a plate near the actual target board. I'd drive back, set up, shoot a handful of shots on the plate to get a feel for the wind... and then immediately shoot my group (or groups) on paper. Had to leave plenty of separation between aiming points on the paper, as our range gets some gnarly cross winds and as you noticed, sometimes things can change significantly between the start of the group and when you finish. So no, you didn't necessarily miss anything (other than maybe the bullseye
).
If the range in question has wind flags up and/or mirage boards, you might want to start learning how to 'read' them
