A few things.
Less is more for case neck cleaning. I tend to try and leave the carbon from firing inside, similar to what many have mentioned. I don't even bother to remove lube inside the neck (I use imperial wax lube, not liquid, though).
A good chamfer is a must. Use a VLD chamfer too, even for non-VLD bullets. This will prevent damage to jackets, and make seating smoother.
If a rifle struggles to stay "in tune," a more forgiving powder with wider nodes will often be more consistent. Extruded temp-insensitive powders tend to have distinct nodes, and need to be in these nodes to perform well. In comparison, some older extruded powders (IE IMR or the N1xx series VV powders) may have wider nodes and be more forgiving.
Some barrels just aren't great shooters, or are more finicky than others. It is possible your barrel just isn't that great, although less likely because it does generate some good groups from time to time.
Another thing to try is shooting off of bags instead of a bipod. A bipod will be more likely to generate vertical dispersion if inconsistent position, or loading of the bipod, is happening.