Not going to work, IMO. Get a better spotting scope or a better scope on your rifle if you're self spotting.
Some of the targets where I shoot are tough... suspended target in front of vegetation at 800 yards. Often the miss can only be identified by the movement of a branch or plant that got clipped by a bullet. You need good glass to see it.
If you're on your own property or are able, get out the weedeater and clear the ground down to bare dirt where the misses would be left and right of the target. That would help.
Edit... If you've got a spotting scope good enough to see bullet holes in cardboard or paper beyond 500 yards, then you've got a spotting scope good enough to stand behind the shooter and watch trace.