Re: How much better is custom?
1) Without worrying about ballistics, as mentioned above, consider the weight of this rig and what you intend to do with it. If random hunting of local fields for yotes where you can drive and setup with minimal walking or mostly targets from prone or bench is the plan, go balls out. But if you have dreams of using a big gun (338) on an expensive elk hunt, you may be pissed after the first day of lugging that big boy around. So this scenario leaves you with an expensive toy and a ruined hunt.
And customs don't have to be heavy, but match/target rifles are typically much heavier than a hunting rifle. It really depends on what you want more. A high level target rifle or a practical hunting rifle.
2) Speaking of practicality, harvesting game at the ranges you are mentioning isn't as easy as some TV shows make it out. Maybe those guys really are that good, but I can't ethically see shooting beyond 600 yards at my current skill level. If I can't get closer, I'm doing something wrong. You probably are a much better shot than me, but without practicing repeatedly beyond 500 yards expecting "game accuracy" at 1000yds or less is setting the bar high. Custom or not, you have to place the bullet in the right spot, not the gun. You mentioned you currently shoot out to 500, how are you doing inside of that range with your groupings?
3) Cartridge- .308 and 338 are worlds apart. It's like going to Africa looking for an Eskimo. There's also about a 60$ difference per 20 shells. In one post you mentioned a .260. If you plan on using a guide service for elk, some outfitters require a minimum cartridge and .260 may fail to qualify. Just food for thought. For all that you're asking 1 gun to do, a .284 or 7mm mag/wsm may better suit your wants/wishes.
4) And you don't have to drop 2-3k on a scope. Weaver, Vortex, and bushnell can satisfy your needs for under a grand or you can pick up a used NF for around 1400$.
Before I went full custom or dropped some serious coin, I'd do more research and then more research just before doing more research. That's not meant to be rude, but based on the OP it sounds like this requires more thought prior to purchasing.