Definitely the #1 question is "how does it shoot?" Since you answered that, then you can move to #2: "Are you satisfied with that?" If the answer is no, then time to mess with it. If you are comfortable with the accuracy, then don't mess with it. Just don't base #2 off of dick measuring on the forums, it has to fall short of what you want for its intended purpose.
That is a less important area of a barrel for a gouge (if there is really such a thing. Across the lands would be a different story. I've shot barrels that looked like a warzone through a borescope, and they were still sub-minute.