I've read where .020 is optimal clearance for accuracy.
Might just be internet hype.
.030" oversize is as close as you want. In tests it made little difference if it was .030" to .060" even .1" over bullet diameter worked surprisingly well. I sometimes make my own muzzle brakes, and they are .030" over, and never use crush washers. Commonly used Crush washers do not allow a perfect square seat, and can force the brake bore off center of the rifle bore, causing some rather ugly runout of the brake. Plus a host of other manufacturing faults can cause misalignment of the two bores. As in the picture above the brake runout was excessive even when butted against the square face of the barrel, and the brake bore, bored out and re faced square. You can machine and thread a barrel stub and check your brake runout in a lathe, add a crush washer and tighten more runout will be added.
That and fouling build up, can get bullet interference if the bullet and brake are too close in diameter. Plus the fact there is little benefit in going closer.