Theorically, if the barrel slugged at .313, the .312 should be ok but 4" at 100 is a bit much for a Finn. You slugged the barrel yourself? slugged it myself
MJ1's suggestion of checking action's screws is good; you can also carefully inspect the crown and bore.
How is the trigger? It could be another factor. the trigger seems pretty good for what it is, not a k31 or 03a3 but pretty good.
If you were to shoot from a sub-MOA rifle, how would you do; sub-MOA. MOA, 2 MOA? my fn shoots half in at 300. my 03 is half minute, highwall quarter minute and bot k31's ar well under a minute. they might do better, but that is with me shooting them.
You may also try various surplus and factory ammo. When you say you tried a couple of loads, do you mean reloads? A couple may not be enough to find what works in your rifle.
You have few more checks to do, before assuming the rifle is a bad shooter.
No matter what, I wish you good luck and hope your rifle will end up being a shooter.
Good shooting.
Ombre noire