This. Plenty of options under $300, that's easy, and they'll do just fine for drilling holes.
I've abused the crap out of my $150 Ryobi drill press from Home Depot and it's still running strong 10+ years later. If you're not a machinist doing specialized work I don't see any good reason to be a drill press snob. It's just a tool.