Majority of time a brake will give you a slight change in POI, will cause you to rezero scope but not change accuracy of load.
Ideally, load would be developed in actual configuration that rifle will be used to fire the ammunition. If you see a huge shift in accuracy of load, might want to make sure you haven't gotten too crazy on torquing down brake to barrel (self timing brake w/locking nut), that brake is not concentric to bore which could be due to threads on muzzle or in brake.
What Milo 2.5 said.