Great place to start, I did the same thing to get into precision shooting. As stated above, ditch the stock... that will be the best bang for your buck as far a upgrades go.
I think the best way to decide is to shoot it, many people will tell you that if you have to install a brake on a .308 you are a wimp.
To me, no matter the caliber, less recoil = better