You're getting great results.
A few years ago I put together a moderate budget mid-range rifle. Aero upper and lower receivers, ARP 20" .223 Wylde, PSA BCG, Geissele SSA-E trigger, MI 15" SP handguard, Luth-AR stock. I'm presently using an Athlon Cronus in a Warne mount.
Initially the receiver wasn't lapped or shimmed. Extension fit was relatively loose. This caused POI to wander. Each shooting session the first group would be great, then shots would open up and cluster in several different places. At the time I didn't have any shim stock, so I used a .0015" feeler gauge blade. It was very snug and required the barrel to be tapped in even after heating. That made a world of difference. The wandering stopped. 5 shot groups with my 73 ELD handloads ran about .6" or a bit under at 100 yards, and they held tight vertical at distance.
Recently, something changed. All along I've had occasional first shot flyers when chambering from bolt lock. They suddenly started happening every time and were much further outside the main group. Also, and unpredictably, the main groups would open way up, then settle back down. Sure seems like something moving that shouldn't. This barrel only has 1000 rounds through it and has never been rapid fired. Apart from a little heat checking near the throat, it looks fine with a borescope.
I noticed that a considerable amount of carbon had built up on the muzzle face. It extended to the very edge of the bore. I removed the flash hider and carefully cleaned off the carbon without touching the edge of the crown. I can't tell for sure if doing this tightened the main group by much. It didn't hurt but also didn't solve the problem. The first shot flyers remained. The first rounds have been going low left by over an inch at 100 yards. I don't think the problem is limited exclusively to the very first round fed, though it has been the most pronounced.
I swapped scopes and mounts to rule out anything going on there. I tried different magazines with no change. I tried hand feeding the first round by inserting the cartridge and then dropping the bolt from about half way. This helped some of the time, but not always. I also swapped in a different bolt carrier in case this was from a gas key alignment issue. No change.
My theory is that the problem stems from not lapping the receiver in combination with using a shim that isn't wide enough. The feeler gauge is only .510" wide, and it was butted up against the extension flange. The extension is smaller in diameter for about .125" rear of the flange (where the barrel pin is located), so effective shim width was only about .385". I'm thinking that over time some slight movement of the extension became more movement. This is only speculation.
Last weekend I removed the barrel. Without a shim, I put the barrel in the receiver and held it up to a light. I could see light all along the right side of the interface. The .0015" feeler wouldn't quite fit in the gap, but it was close. I lapped the receiver (carefully, with receiver vertical in a vise and turned by hand) until I had bare aluminum all around. Probably took off a couple thousandths. A friend has shim stock in various thicknesses. We cut a piece of .001" about .80" wide and long enough to wrap around, leaving a gap just slightly wider than the barrel pin. Heated with a torch and slipped in the barrel. Seems good. Before greasing the barrel nut I snugged it down a little and then removed it. I couldn't see any light showing between receiver and extension. Greased and torqued.
This MI handguard is the large diameter model. It has an aluminum barrel nut that must be indexed. Before lapping it timed at about 45 ft/lb. I didn't have any thin barrel nut shims or much assortment. I ended up with one about .025" thick, which timed at about 32 ft/lb. I'm going to try it at this, but I've also ordered some .001" shims in case I wan't to increase the torque.
The thick barrel nut shim has the barrel nut set out far enough that the handguard doesn't quite touch the receiver. I had always assumed that contact was ideal. It's good to know that things may be better with a small gap.
As for gas tube alignment, it was pretty good to start with. I made some very minor adjustments, but it's really hard to tell when it's aligned as well as possible. The tube has a lot of play within the receiver hole, and position of the bolt carrier makes a difference. When checking, I pushed the carrier up simulating magazine pressure. I can hear a slight "tink" of contact being made, but the tube moves only very slightly when the gas key slides over it. It doesn't seem to be exerting pressure when the carrier is fully forward. I could try further improving the alignment, but I don't know if I can get it much better.
I'm going to shoot tomorrow to see if lapping and re-shimming has helped. If not, I'm leaning toward trying a different upper receiver. MURs seem to be out of stock everywhere (I doubt this SP model handguard would fit). So are BCM. Maybe a Mega. I can't afford an expensive billet. I also have a new 20" ARP barrel, but at this point I don't think this is a barrel issue.
Pics below are of the carbon on the muzzle. Note that in the before pic facing the muzzle that the carbon isn't completely even. I wondered if pieces flaking off would cause a "false crown" and throw shots until the carbon had evened out again. It's something to keep an eye on. To clean, I soaked with KG-1 and carefully scraped. I didn't touch the edge of the crown.