I have used a re-profiled small diameter solid carbide boring bar for years to clean up the bolt face, but I'd recommend against futzing with getting under the extractor.
The riveted extractors are easy enough to replace with a little bit of time, and the non-riveted versions even more so.
If you need a rivet to replace and can't find any in stock, I'll send you one.