Re: OK to remove the expander ball?
The only issue will arise if you have a dinged case mouth. As careful as I try to be, I always have a few cases that get the mouth flattened a tad, either during cleaning transport or whatever. If you size without an expander, there isn't anything in your process to round the mouth back out except your bullet when you seat it. I don't like this idea, so....
I tune the expander in all of my dies to not produce runout(making sure it's centered), and I spin all the expanders down on some emory cloth til they provide extremely little to no contact with the case mouth on the extraction side of the sizing operation. This insures that I don't end up with a case with a mouth slightly out of round, that then depends on the bullet to true it up. By turning my expanders down such as I do, it does not alter the neck tension of the cases if they were to be sized without the expander, which I suppose, has been reduced to the roll of a 'case mouth truing device' as opposed to an actual expander.
Just an option for you, try it, you may like it.