• Watch Out for Scammers!

    We've now added a color code for all accounts. Orange accounts are new members, Blue are full members, and Green are Supporters. If you get a message about a sale from an orange account, make sure you pay attention before sending any money!

Reloading Art vs Scientific Method

14DFASniper

Private
Minuteman
Jun 15, 2012
11
0
51
Hello,

I am interested in learning more about the Art vs Science of reloading. It seems that most people who reload "tinker" a little bit looking for the perfect combination that matches their rifle.

I'm wondering if anyone has used response surface full factorial modeling to identify the "perfect" bullet combinations for the most desirable results?

For those of you who don't know what RSM modeling is here is a breif description. Basically, you define the control input factor ranges like bullet grains, powder grains, seating depth, shooting distance to target, head space...basically anything you want that you can "control"

Then you identify the "names" of measurable responses that are make up the desirable results like precision (ie averaged shot group size), accuracy ( distance from the bulls eye at the defined shooting distance), muzzel or chronograph velocity, etc.

From there you generate a series of test of your input factors and fire your rifle, measure and record your results. From there you put the data into a software program that generates a 3D model and prediction profiler.

Think of it like a cube with a topographical image of a mountain that you can look at from all sides. You can then have the software tell you what your optimum load combinations are and how big of range you have before performance deminishes at various combinations. It will also predict how your rifle will perform at virtually any combination within the cube size, and how All of the reponses combinations will perform. So If you have a 175 grain bullet with x seating depth, Y grains of powder, Z distance to target you will know what your accuracy, group size and muzzel velocity will be for that load combination befor you even shoot the round or even build it at your reloading bench.

It is extreamly important to know that garbage in equals garbage out. So you have to be an obsessive compulsive nut like myself to generate a good model. Everything must be measured and matched accuratly while you go through the process of building the loads and recording the results.

However, the beauty of it is this. You generate a model that maps out all of your parameters with minimum shots fired. So a basic model can be as low as lets say 20 shots but the model will be equillivant to 3,340 shots or something like that. The more levels you have...obviously the more shots you have to make. So a huge model in triplicate shots would be something like 300.

This is something I'm going to do with my FNH SPR 300 WSM and my .308 once I can afford a progressive reloader.

If one or two of you would like to try the modeling method let me know. I would be willing work with you to generate the model if you are willing to load and measure the results with extream precision. (garbage in will be garbage out I promise you that) I cannot do any more than one or two because I don't want this to be "work". I am just "curious" how well it will match up and thought it might be interesting and useful for some of you as well.

Once the model is built I can send you the results and model in pdf format with the "ideal" input factor settings. Then you will need to go out and shoot at the recommended setting to prove out the model and report back how well your system performed.



 
Re: Reloading Art vs Scienctific Method

Very interesting, you have obviously put a lot of thought into it. There is a lot of art that goes into reloading, science as well but coming close to a conclusion before you buy a single component is a time and money saver. There are a handful of good load programs out there but many have their limitations, Quickload is one that many here use.

As far as "precision", I would advise picking up a single stage press. A single stage is much cheaper and they tend to produce more consistant controled results because you are breaking up 3-4 stages and performing them one at a time. I use a few Dillon 550 progressives for volume loading and all of my precision loads are cooked up on a Forester Coaxial single stage press.

BTW, welcome to the Hide



Kirk R
 
Re: Reloading Art vs Scienctific Method

This sounds very intriguing to say the least. How practical/functional it is I might be a little skeptical. Only reason I say that is how many variables go in to factoring ballistics accurately and consistently, but then again I am very much a simple minded man and definitely not OCD enough to get good results. I am though, Very Interested in following your research/results. When you get the volunteers keep us posted as you move through it.
 
Re: Reloading Art vs Scienctific Method

I used to shoot HP with a scientist that approached loading in a similar fashion. His quest never ended.
 
Re: Reloading Art vs Scienctific Method

I am with Cap Kirk on the press. I have both single stage and progressive presses and the single stage is far more precise. If really controlling all variables possible you cannot use a progressive to full advantage anyway. They are great for kicking out a bunch of pistol ammo in short time but do not allow for all the brass preparation processes.

Your process sounds like a program running a statistical ANOVA (ANalysis Of VAriance), which is a very sound approach. There are some articles around that treat the subject fairly well but not to the point of a "cookbook". I wish you success and am very interested to hear of your effort, but for success you will have to arbitrarily fix some variables as constants. Otherwise, you will be lost in confusion. Even then, you are going to have to load a hell of a lot more than 20 rounds.

There is considerable random variation in every variable within the system and there is a LOT of variables, so your problem is not trivial. NOT the least of these variables is the human factor in every step from pulling the press handle to squeezing off the test shots. Even a well experienced handloader who is an expert marksman and has PhD in statistics would find the task a considerable challenge.
 
Re: Reloading Art vs Scienctific Method

At my local club I watch both the "Artists" and the "Scientists" shoot. It's amazing how many more Records and Trophies are held by the "Artists".
 
Re: Reloading Art vs Scienctific Method

"...modeling to identify the "perfect" bullet...Once the model is built I can send you the results and model in pdf format with the "ideal" input factor settings."

Assume you're new to reloading?
 
Re: Reloading Art vs Scienctific Method

This sounds pretty cool. Do you already have access to the volumes of powder burn rates, case capacities, ballistic coefficients, etc?
 
Re: Reloading Art vs Scienctific Method

I can't wait. <yawn>
 
Re: Reloading Art vs Scienctific Method

I just followed this approach, developing a load for my 300wm. I read Dan Rorabaugh's 2001 article on DOE (Design of Experiments), obtained an excel plug-in (Minitab 16 is the cheapest), and went to work. The big advantage of following this approach is that you can completely determine the effects of each factor on accuracy (or other variable like MV) and KNOW you've found the optimum load - there will be nothing better for your rifle. Here's how I did it.

I first shot a single-shot-per-grain-load ladder test across a wide range of powder loads, both to see what powder loads gave a node, and check for pressure signs. The Berger OTM 230 is a long bullet, so I chose H4381 which would take up less space than an H1000 load.

I wanted to check 3 factors (variables) and 2-3 levels for each factor:
primer (CCI 250 and Fed 215 Match)
seating off lands (.020, .030, .040)
powder load (71, 71.2, 71.4 gr)

My goal would be to find the levels for each factor that would minimize the group size.

If you ran a full test, that's 18 "experiments", and you probably want to run it at least twice to remove any human-induced outliers. Fortunately, there are statistical methods (Taguchi, for those who care) for sampling that allow you to fully investigate effects with a smaller but well-designed sample. I ended up with 9 different combinations.

My goal/control was group size, of course. I spent a careful evening loading 6 rounds for each of the 9 combinations, and printed out 18 targets (2 replications for each factor combination). I also loaded up 3 rounds from one group to shoot the cold bore shots and zero on paper. I shot at 300 yds, on almost windless days.

The test runs were shot in randomized order (ie a single shot from group 7, the next from 9, etc) at their respective targets. This was repeated until I had built a group of 3 shots for each combination (about 3 hours, waiting for barrel to cool). I then replicated the entire test on a different day. This is important to reduce the effects of the human variable (as mentioned above) This gave me two 3-shot groups for each of the 9 combinations, 54 shots in all. I used OnTarget to scan and measure group size.

The results were definitive, group size remaining fairly consistent across the two replications. The groups ranged from .4 to up to 1.16 MOA. As OlTexasBoy mentions above, one part of the results is ANOVA. More importantly, the statistical package generated the quantitative model for each factor.

The model showed that seating depth had by far the biggest effect, followed by powder level and primer. That seating had more effect than powder was perhaps expected as I had already determined a powder load node.

The modelling is the MOST important element of this method. The load I identified (and subsequently verified) was actually NOT one I had actually tested! Remember I only shot 9 of the 18 total possible combinations? The combinations I used were enough to fully characterize the effects of each factor. Using the model, I easily found the optimum combination, and the model predicted a group size of .36 MOA- smaller than the smallest group actually shot. This would be interesting...

I loaded up 6 more rounds with that combination and shot (2) 3 shot groups. The two groups measured .319 and .377 MOA; the .348 average is very close to the .36 predicted. I have my load, and know with certainty there is no better.

This method takes a LOT of time, a LOT of shots (10 for the ladder, 3 for pre-test warm-up, 54 for the test, 6 for confirmation), and very careful accounting (I marked each bullet with its group number, clipboard with checklist for each shot). If you want to investigate more than 3 factors, the efficiency rises fast (sample size vs total factorial) as the OP points out.
 
Re: Reloading Art vs Scienctific Method

Hello Paul,

Very cool information! Not perhaps everybody's cup of tea, but interesting to say the least. I've seen references to using DOE for load testing and such in the past, but not much in the way of stuff to bridge the gap between engineers trained in such, and the rest of us (even the 'technically oriented' ones). As an example, I've taken a basic stats class, actually enjoyed it, but it didn't cover much in the way of tools for handling this sort of data.

73 shots may seem like a lot... but I'm pretty sure I've burned way more than that on more than one occasion - just as I've also 'stumbled' upon a magic load in considerably less.

Out of curiosity... have you ran across this site before:

http://statshooting.com/

Thought it might be of interest / relevance to the topic at hand, since you mentioned using OnTarget, etc.

I'd definitely be interested in more (intelligent) discussion on this topic. If it needs to move off-forum to private messages or email to avoid the trolls, so be it.
 
Re: Reloading Art vs Scienctific Method

Paul and Memilanuk,

I have often been told I look like a troll, but promise I am not. I recently bought a rifle and need to work up a load. I took a couple of statistics classes back during the Jurassic Age. I am too ignorant to construct and analyze the experiment, but definitely understand the possibilities.

If you take this offline I would appreciate being included. Paul’s write up is very well done however and I doubt too many would want to make themselves look foolish by berating it in public. I think quite a few folks on the forum may stand to gain.

Of course the original question was; “art or science”. Do not most worthwhile endeavors combine the two?
 
Re: Reloading Art vs Scienctific Method

OlTexasBoy,

Wasn't referring to you as one of the trolls. They know who they are
wink.gif


Like I mentioned, it may not be everyone's cup of tea, and if someone has a system worked out that gets them the results they want i.e. good enough accuracy to get out there and work on other things like range estimation and wind reading, then that may be where their interest in this sort of thing ends. Others find the whole load testing thing to be fascinating in and of itself. To each their own.

As far as using RSM vs. DOE... no idea if one would be better than the other. I have seen/heard of DOE before; found a link to an old (2001) paper on doing load development via DOE - Paul had the same pdf floating around. When I get home tonight I'll try to post a link via Dropbox for those interested. It's an interesting read. One take could be that the guy wasted an awful lot of rounds to confirm that .224 52gn SMKs over almost 27 gn of W748 shoot really well in a .223 Rem. The other take could be that not surprisingly, the DOE results ended up pointing almost *exactly* at a 'known good' load that works in a lot of guns - with a few individual tweaks like which primer and what seating depth and which brand of case worked best in *that* rifle.
 
Re: Reloading Art vs Scienctific Method

meme, Amigo. Maybe it's not so obvious to whom you are calling a troll? If you think it's me, you can kiss my fucking ass.

Take a look at the thread, "unsafe reloader?" and shed a tear or two at the attitude of a lot of our hide members, which I think is a shame, but take note of who thinks even reading]/u] is for chumps. Now how do you think these people are going to react to this extremely long and involved experimenting talked about in this thread.

Listen, I have been active on the Internet for 16 years, own a message board and have moderated six others. Nobody has ever suggested that I might be a troll. Yeah, highly opinionated, and damned proud of it.

On the other hand, maybe you were thinking of somebody else and if so, I apologize. BB
 
Re: Reloading Art vs Scienctific Method

<div class="ubbcode-block"><div class="ubbcode-header">Quote:</div><div class="ubbcode-body">Nobody has ever suggested that I might be a troll.</div></div>

Really? And your earlier 'contribution' to the thread would fall under what classification...?
 
Re: Reloading Art vs Scientific Method

<div class="ubbcode-block"><div class="ubbcode-header">Quote:</div><div class="ubbcode-body">Really? And your earlier 'contribution' to the thread would fall under what classification...?</div></div>

IT'S CALLED A FUCKING <span style="text-decoration: underline">COMMENT</span> AMIGO. DO YOU UNDERSTAND WHAT THAT IS? IT'S NOT POSITIVE OR NEGATIVE, IT'S A COMMENT, AND IF YOU DON'T LIKE IT; SEE ABOVE SUGGESTION.
 
Re: Reloading Art vs Scientific Method

Anywho, back to the topic at hand...

If the original poster hasn't been completely run off by now, it would be nice to hear back from ya. People are interested...
 
Re: Reloading Art vs Scientific Method

To the OP, Monte, and most others,

An interesting and definitely viable thread. I, for one, hope that the patience and perseverance needed to continue on with this quest, as well as this discussion, does not waver.

There are actually many here, who are interested in learning new things and methods. It is the shit-head drama-queens and soap-opera enthusiasts who must be disregarded and ignored.

My request: Please, keep it up, keep it going, and continue to share this here. Every class must have it's clown. I think it's a law, or something.
 
Re: Reloading Art vs Scientific Method

I asked if the OP has been reloading long because I've been doing this stuff long enough to firmly believe there are too many imponderables that will ruin any happy arm chair theoritcal accuracy pursuits. I doubt the value of computer analysis of the 'perfect cartridge' and believe that to whatever extent the chase may succeed the results would only apply to the tested firearm and loads. But I may be wrong, so, whatever, if it's fun then everyone should happily chase whatever rainbows they wish. (Even if it puts some people to sleep!)

-------------------------------

Just a thought; has anyone but me noticed how many troll types just wanting to share their knowledge and vast intellect will immediately get quite touchy, crude and personally abusive if anyone disagrees with their immature comments?

BB: "I can't wait. <yawn>" "Kiss my ... (whatever)." Including an impresive list of impressive credientals!
smile.gif
??
 
Re: Reloading Art vs Scientific Method

<div class="ubbcode-block"><div class="ubbcode-header">Quote:</div><div class="ubbcode-body">I asked if the OP has been reloading long because I've been doing this stuff long enough to firmly believe there are too many imponderables that will ruin any happy arm chair theoritcal accuracy pursuits. </div></div>

As have a lot of us here. I may be wrong, but my understanding is that with DOE and similar techniques you control the inputs you want to vary, stabilize the ones you can and attempt to randomize the effects of the ones you can't. A very rough analogy might be how the OCW load development method uses a round-robin firing pattern in an attempt to spread any effects of barrel heat or fouling over all the groups rather than just the latter ones. With DOE the test groups would be fired in a random sequence for similar reasons. Will it completely eliminate the effects of things like a rain squall coming in during your test cycle, or a pulled bad shot? Probably not. The real world does have a nasty habit of intruding whether we wish it or not
wink.gif


I'd looked at this sort of thing years ago when a thread came up on nationalmatch.us where the poster (the author of the 'DOE for Reloading' article linked above) put forth his process. I was all excited about it up until I saw the cost of the software required. Unless you're willing to either do the math by hand or are a decent engineer/programmer yourself, the packages to do this sort of thing are not cheap - as in $500-1500+ for a single user license. There are some 'free' options such as a GUI plug-in for the R scripting language (r-project.org) which in turn has a DOE plug-in available... but that starts becoming a pretty big rabbit hole to go down in order to 'optimize' your load. The other alternative is to find someone who as access to a licensed version professionally and get them to do a run for you (as the OP mentioned).

If there is some other option that is more accessible to the 'average' user, preferably free or open-source (stand-alone app, plug-in for Open Office, etc.) I'm all ears.
 
Re: Reloading Art vs Scientific Method

Interesting subject here with a lot of good information for those that wish to pursue scientific handloading to that degree.

I'll forgo the DOE and the rest of the Six Sigma activities stick with OCW.


OFG
 
Re: Reloading Art vs Scientific Method

Trying to keep the ball rolling here, even though it kind of feels like I'm talking to myself...
wink.gif


Been spending some time going through the documents mentioned earlier in this thread, along with the NIST Engineering Stats Handbook section on 'Process Improvment', aka Experimental Design. The latter isn't exactly 'light' reading, although its a lot clearer than some other documents I've looked at.

So far about the only FOSS option for doing anything like DoE is R, a scripting language/ environment for statistical analysis. There is a GUI interface available (R-commander) and there is a plug-in for that (RcmdrPlugin.DoE) that simplifies a lot of the scripting/command-line work otherwise involved in using R for this sort of thing. Not enough, though, unless you're fairly dedicated. Here is a page for the RcmdrPlugin.DoE; note that there is a link for a 50+ page tutorial with data files on the basics of using Rcmdr for DoE analysis. If you do decide to diddle with it, bear in mind that the author is German, and uses commas instead of decimal points and semicolons instead of commas - requires a little manual sanitizing of the example CSV files to make everything work.

I'm getting closer to thinking that those DoE software packages, even the Excel plug-ins, might actually be worth the money if one wished to pursue this seriously...

Anywho... one of the things that I've come away from all this reading and tinkering with software so far is this: if you want to determine what factors have the most effect out of a bunch of possible factors, and can reduce them to two levels like 'brass A or B' or 'primer A or B' or 'seating depth 1 or seating depth 2'... then screening or two-level fractional factorial experiments to determine which one of those factors has the most effect would work well. Thing is... most of us who've been doing this for some time have pretty much figured that sort of thing out (sometimes the hard way!) already. If you want to do something along the lines of what the original poster mentioned, i.e. response surface testing, where you would be testing for interactions between multiple factors with multiple (more than 2) levels... that might be something where DoE would come in handy. Whether it would be feasible... is another question entirely.

So, with that, I have a question for the experienced traditional loaders out there reading this... lets say you're fine-tuning a load. You've got the powder charge weight pretty close to dialed in, and you've got the seating depth pretty much where you think it should be (assuming not limited by magazine length here). You've tried more than one primer, differing neck tensions, etc. Most people, if they tinker with those things at all, tend to do them one at a time - get the seating depth where the gun likes it, then adjust the powder charge, then maybe neck tension, then maybe back off a little and try a different primer and work back up the powder charge. Back n forth until you get everything dialed in... or your barrel dies
wink.gif
Different people may do these or other steps in slightly different ways, but you get the general idea. Everything is tested 'one factor at a time', the way we learned in grade school science class. But...

...did you ever wonder if there is some kind of interaction between those parameters that might be affected not just by changing one, but more than one of them at a time? Most of us get pretty good results doing things the traditional way... maybe we get lucky and stumble onto the best combination anyways. Testing for the interactions that could optimize the results may be a royal PITA, for sure. Gotta wonder if it might be worth it, though...

Monte
 
Re: Reloading Art vs Scientific Method

Art or Science, it all takes time & money. Most of us are limited on both. I'm old school. Reloading is a craft that can be polished through years of discipline and knowledge. Each of us that take reloading seriously are artisans in our own way. That said, we must remember that we really love to shoot so we must keep it within reasonable expectations. We live in an imperfect world, yet we strive for perfection.
 
Re: Reloading Art vs Scientific Method

I think the question comes down to end goals. If you're a benchrest shooter with the opportunity to shoot sighters and pre-determine wind before you shoot record shots, and you measure success only when you get below .2 MOA, its probably worth the time, expense and barrel life. Also, BR shooters are an analytic bunch...

I was able to get to sub .5 MOA, which in a tactical environment is probably sufficient given the other unknowns (range, wind etc) and challenges (ie non-prone shooting, movers).

I used the free trial period of software to get my tests done. I do think there's enough value in it to purchase the s/w the next time I need to do it. The barrel life and ammo saved will pay for it (esp for 300wm).

As Monte writes, testing multiple variable changes simultaneously can be more valuable than a one-at-a-time approach, which can easily miss local minima. The fact that DOE models the variables, allowing you to find optimum solutions that haven't even been tested (as I experienced)is a big plus.

I also look at it as another excuse to go shooting. That can't be bad!
 
Re: Reloading Art vs Scientific Method

I think this approach would be handy for determining bullet seating depth and charge weight. I was thinking about doing an experiment design like this one. For a slightly more complicated test, I would add in bullet manufacturer for similarly designed bullets. Understand that in a given bullet design, there are a lot of different discreet factors such as bearing surface, bullet weight, overall length, nose radius, shape (secant vs. tangent), etc.

On of the issues that you might run into is the robustness of the local minimum. You don't want to design around a local minimum, have some throat erosion or variance in bullet dimensions and end up falling off of a cliff.
 
Re: Reloading Art vs Scientific Method

Carter,

Great question!

DOE s/w "marginal means" plots show the shape of the minimum vs the levels of the factor; you can judge the sensitivity of the output to the factor from that.

Adding in numerous factors (especially 2 level ones like bullet A and bullet B) do not require many more runs.
 
Re: Reloading Art vs Scienctific Method

Thanks for the advice my friend. I agree that a single stage will produce a higher precision bullet.

I will want a single stage for my 300 WSM and 308 precision rifle.
 
Re: Reloading Art vs Scienctific Method

You are correct.. probably not practical, but it is actually fun to see the data and know where you are optimum and where you are not. You might be 0.5 MOA at 100 yards but at 800 yards you just might not be doing as well as you could be. How do you know until you "test" it and measure. Makes it a fun process just like reloading itself.
 
Re: Reloading Art vs Scienctific Method

<div class="ubbcode-block"><div class="ubbcode-header">Originally Posted By: OlTexasBoy</div><div class="ubbcode-body">I am with Cap Kirk on the press. I have both single stage and progressive presses and the single stage is far more precise. If really controlling all variables possible you cannot use a progressive to full advantage anyway. They are great for kicking out a bunch of pistol ammo in short time but do not allow for all the brass preparation processes.

Your process sounds like a program running a statistical ANOVA (ANalysis Of VAriance), which is a very sound approach. There are some articles around that treat the subject fairly well but not to the point of a "cookbook". I wish you success and am very interested to hear of your effort, but for success you will have to arbitrarily fix some variables as constants. Otherwise, you will be lost in confusion. Even then, you are going to have to load a hell of a lot more than 20 rounds.

There is considerable random variation in every variable within the system and there is a LOT of variables, so your problem is not trivial. NOT the least of these variables is the human factor in every step from pulling the press handle to squeezing off the test shots. Even a well experienced handloader who is an expert marksman and has PhD in statistics would find the task a considerable challenge.
</div></div>

The number of rounds depends on how many factors and levels we are building into the model. A really large model will be better to run a fractional factorial than a full response surface model. ANOVA would be interesting to measure the statistical differences, we can do that too with the same data that is acquired. But the primary objective would be to minimize the number of factors starting out and generate a simple model at first. I agree on the "shooter" expertise but we can actually measure that with the replicates of some of the center points of the first model.

Challenge - yes. If it wasn't fun taking on a challenge then I wouldn't do it. = )
 
Re: Reloading Art vs Scienctific Method

<div class="ubbcode-block"><div class="ubbcode-header">Originally Posted By: memilanuk</div><div class="ubbcode-body">Anywho, back to the topic at hand...

If the original poster hasn't been completely run off by now, it would be nice to hear back from ya. People are interested...</div></div>

My apologies to those of you who are interested and waiting for a reply. I've been working 12-14 hour days so I don't have much free time to surf the internet. But, that is not to say I'm not interested in learning. I am not currently a reloader yet. It is new to me. I have been doing a lot of reading on the subject when time permits. Bottom line is I'm pretty confident a person can "optimize" a lot of things quicker by applying, measuring, and analyzing the science up front. By doing that, a person can learn and understand the art / skill faster.
 
Re: Reloading Art vs Scienctific Method

<div class="ubbcode-block"><div class="ubbcode-header">Originally Posted By: Deadshot2</div><div class="ubbcode-body">At my local club I watch both the "Artists" and the "Scientists" shoot. It's amazing how many more Records and Trophies are held by the "Artists".

</div></div>

Agreed, but the best people I work with are Artist who are also Scientist. It's not about one vs the other. It's about being both and working towards excellence efficiently.
 
Re: Reloading Art vs Scientific Method

<div class="ubbcode-block"><div class="ubbcode-header">Originally Posted By: memilanuk</div><div class="ubbcode-body">Trying to keep the ball rolling here, even though it kind of feels like I'm talking to myself...
wink.gif


Been spending some time going through the documents mentioned earlier in this thread, along with the NIST Engineering Stats Handbook section on 'Process Improvment', aka Experimental Design. The latter isn't exactly 'light' reading, although its a lot clearer than some other documents I've looked at.

So far about the only FOSS option for doing anything like DoE is R, a scripting language/ environment for statistical analysis. There is a GUI interface available (R-commander) and there is a plug-in for that (RcmdrPlugin.DoE) that simplifies a lot of the scripting/command-line work otherwise involved in using R for this sort of thing. Not enough, though, unless you're fairly dedicated. Here is a page for the RcmdrPlugin.DoE; note that there is a link for a 50+ page tutorial with data files on the basics of using Rcmdr for DoE analysis. If you do decide to diddle with it, bear in mind that the author is German, and uses commas instead of decimal points and semicolons instead of commas - requires a little manual sanitizing of the example CSV files to make everything work.

I'm getting closer to thinking that those DoE software packages, even the Excel plug-ins, might actually be worth the money if one wished to pursue this seriously...

Anywho... one of the things that I've come away from all this reading and tinkering with software so far is this: if you want to determine what factors have the most effect out of a bunch of possible factors, and can reduce them to two levels like 'brass A or B' or 'primer A or B' or 'seating depth 1 or seating depth 2'... then screening or two-level fractional factorial experiments to determine which one of those factors has the most effect would work well. Thing is... most of us who've been doing this for some time have pretty much figured that sort of thing out (sometimes the hard way!) already. If you want to do something along the lines of what the original poster mentioned, i.e. response surface testing, where you would be testing for interactions between multiple factors with multiple (more than 2) levels... that might be something where DoE would come in handy. Whether it would be feasible... is another question entirely.

So, with that, I have a question for the experienced traditional loaders out there reading this... lets say you're fine-tuning a load. You've got the powder charge weight pretty close to dialed in, and you've got the seating depth pretty much where you think it should be (assuming not limited by magazine length here). You've tried more than one primer, differing neck tensions, etc. Most people, if they tinker with those things at all, tend to do them one at a time - get the seating depth where the gun likes it, then adjust the powder charge, then maybe neck tension, then maybe back off a little and try a different primer and work back up the powder charge. Back n forth until you get everything dialed in... or your barrel dies
wink.gif
Different people may do these or other steps in slightly different ways, but you get the general idea. Everything is tested 'one factor at a time', the way we learned in grade school science class. But...

...did you ever wonder if there is some kind of interaction between those parameters that might be affected not just by changing one, but more than one of them at a time? Most of us get pretty good results doing things the traditional way... maybe we get lucky and stumble onto the best combination anyways. Testing for the interactions that could optimize the results may be a royal PITA, for sure. Gotta wonder if it might be worth it, though...

Monte </div></div>

You understand. Good. I'm pretty sure it will be worth it. I've found a few people who have done this before with success. But they approached it a bit differently than the way I'm thinking.
 
Re: Reloading Art vs Scientific Method

<div class="ubbcode-block"><div class="ubbcode-header">Originally Posted By: Carter Mayfield</div><div class="ubbcode-body">I think this approach would be handy for determining bullet seating depth and charge weight. I was thinking about doing an experiment design like this one. For a slightly more complicated test, I would add in bullet manufacturer for similarly designed bullets. Understand that in a given bullet design, there are a lot of different discreet factors such as bearing surface, bullet weight, overall length, nose radius, shape (secant vs. tangent), etc.

On of the issues that you might run into is the robustness of the local minimum. You don't want to design around a local minimum, have some throat erosion or variance in bullet dimensions and end up falling off of a cliff. </div></div>

Yes, this would work for all of those in the same model.
 
Re: Reloading Art vs Scientific Method

<div class="ubbcode-block"><div class="ubbcode-header">Originally Posted By: Paul Stafford</div><div class="ubbcode-body">Carter,

Great question!

DOE s/w "marginal means" plots show the shape of the minimum vs the levels of the factor; you can judge the sensitivity of the output to the factor from that.

Adding in numerous factors (especially 2 level ones like bullet A and bullet B) do not require many more runs. </div></div>

Correct.
wink.gif
 
Re: Reloading Art vs Scientific Method

<div class="ubbcode-block"><div class="ubbcode-header">Originally Posted By: memilanuk</div><div class="ubbcode-body"><div class="ubbcode-block"><div class="ubbcode-header">Quote:</div><div class="ubbcode-body">I asked if the OP has been reloading long because I've been doing this stuff long enough to firmly believe there are too many imponderables that will ruin any happy arm chair theoritcal accuracy pursuits. </div></div>

As have a lot of us here. I may be wrong, but my understanding is that with DOE and similar techniques you control the inputs you want to vary, stabilize the ones you can and attempt to randomize the effects of the ones you can't. A very rough analogy might be how the OCW load development method uses a round-robin firing pattern in an attempt to spread any effects of barrel heat or fouling over all the groups rather than just the latter ones. With DOE the test groups would be fired in a random sequence for similar reasons. Will it completely eliminate the effects of things like a rain squall coming in during your test cycle, or a pulled bad shot? Probably not. The real world does have a nasty habit of intruding whether we wish it or not
wink.gif


I'd looked at this sort of thing years ago when a thread came up on nationalmatch.us where the poster (the author of the 'DOE for Reloading' article linked above) put forth his process. I was all excited about it up until I saw the cost of the software required. Unless you're willing to either do the math by hand or are a decent engineer/programmer yourself, the packages to do this sort of thing are not cheap - as in $500-1500+ for a single user license. There are some 'free' options such as a GUI plug-in for the R scripting language (r-project.org) which in turn has a DOE plug-in available... but that starts becoming a pretty big rabbit hole to go down in order to 'optimize' your load. The other alternative is to find someone who as access to a licensed version professionally and get them to do a run for you (as the OP mentioned).

If there is some other option that is more accessible to the 'average' user, preferably free or open-source (stand-alone app, plug-in for Open Office, etc.) I'm all ears. </div></div>

I use a professional version of JMP software which is like $1200 per year. You can pretty much come to the same conclusions by using the common every day excell program and running the same combinations of variables to produce multiple linear regression plots.
 
Re: Reloading Art vs Scientific Method

This is a very intriguing title on this post. Art vs. Science. Never really thought about it that way. I guess the art part is reloading teqnique and proper shooting skills and science would be ballistic data. Hell, all I do is look at my ogive lengths in my data book and start loading them up. I guess you could say I really don't over think it.
 
Re: Reloading Art vs Scientific Method

Coming from a scientific background myself, I would love to see someone apply this sort of scientific methodology to reloading.

That said, I don't think it will be worth your while, other than for the pure pleasure of the intellectual pursuit.

There is a certain amount of fundamental science and statistical method employed by all experienced reloaders. Ladder testing, OCW testing, and the like have their basis in a deeper understanding of how bullets, powder, and rifle interact, and are not purely artistic "trial and error." However, even with this base amount of science, shooters are already at the brink of what is technically feasible. Benchrest shooters routinely have groups in the "zeroes" at 100 yards, and most of them are not scientists.

I think the problem with approaching reloading in this way is that shooting is an art, not a science. You can produce ammunition in a rigorously scientific fashion, but if you can't call wind, or squeeze the trigger properly, you will get lousy groups. Likewise, as with all statistical models, garbage in = garbage out, and I don't think there are many individuals out there that shoot so well and so consistently that you could base accurate statistical models on their group sizes without a large sample size for each discrete variable. In other words, if 43gr of powder yields a .3moa group, but 43.1gr of powder yields a .4moa group, you have to consider whether your model is powered to detect that difference (and that's making the huge asusmption that the difference is real, not the result of shooter variation). Finally, there are a virtually infinite number of variables that you could add to your model - where do you stop? Powder weight and bullet weight are but a small fraction of the things you can look at. Brand of brass, neck tension, how recently the brass was annealed, brand and type of primer, primer seating depth, size of flash hole, flash hole deburring, bullet design, individual bullet weight variation, length of bearing surface of the individual bullet, presence and degree of meplat trimming, rifle twist rate, rifle throat erosion and diameter, amount of jump to the lands, barrel length and profile... the list goes on and on. Then you still have to control for external ballistic factors to be able to compare one test's results to the next: elevation, humidity, density altitude corrections, shooting angle, WIND, etc. Some of these minute variables won't make a difference at 100yds, but they might at 1500 - do you have the space to test them?

In the end, I believe that some degree of science is important to reloading. However, at some undetermined point, the effects of your shooting art begin to outweight the effects of your reloading science (ie, your brand of primer probably doesn't matter if you can't hold quarter-MOA anyway). No one knows where this point is (and it varies for every shooter). I think, ultimately, you apply some basic statistical methods, try a couple brands of popular bullets, and find a load that you can shoot well out to whatever distance you like to shoot. This may not result in the "best possible ever" load for your rifle, but it doesn't take hours and hours of experimentation and number crunching either - hours which would likely be better spent at the range, perfecting the "artistic" side of your shooting.
 
Re: Reloading Art vs Scientific Method

<div class="ubbcode-block"><div class="ubbcode-header">Originally Posted By: Conqueror</div><div class="ubbcode-body">Coming from a scientific background myself, I would love to see someone apply this sort of scientific methodology to reloading.

That said, I don't think it will be worth your while, other than for the pure pleasure of the intellectual pursuit.

There is a certain amount of fundamental science and statistical method employed by all experienced reloaders. Ladder testing, OCW testing, and the like have their basis in a deeper understanding of how bullets, powder, and rifle interact, and are not purely artistic "trial and error." However, even with this
base amount of science, shooters are already at the brink of what is technically
feasible. Benchrest shooters routinely have groups in the "zeroes" at 100
yards, and most of them are not scientists.


What this this guy said

I think the problem with approaching reloading in this way is that shooting is an art, not a science. You can produce ammunition in a rigorously scientific fashion, but if you can't call wind, or squeeze the trigger properly, you will get lousy groups. Likewise, as with all statistical models, garbage in = garbage out, and I don't think there are many individuals out there that shoot so well and so consistently that you could base accurate statistical models on their group sizes without a large sample size for each discrete variable. In other words, if 43gr of powder yields a .3moa group, but 43.1gr of powder yields a .4moa group, you have to consider whether your model is powered to detect that difference (and that's making the huge asusmption that the difference is real, not the result of shooter variation). Finally, there are a virtually infinite number of variables that you could add to your model - where do you stop? Powder weight and bullet weight are but a small fraction of the things you can look at. Brand of brass, neck tension, how recently the brass was annealed, brand and type of primer, primer seating depth, size of flash hole, flash hole deburring, bullet design, individual bullet weight variation, length of bearing surface of the individual bullet, presence and degree of meplat trimming, rifle twist rate, rifle throat erosion and diameter, amount of jump to the lands, barrel length and profile... the list goes on and on. Then you still have to control for external ballistic factors to be able to compare one test's results to the next: elevation, humidity, density altitude corrections, shooting angle, WIND, etc. Some of these minute variables won't make a difference at 100yds, but they might at 1500 - do you have the space to test them?

In the end, I believe that some degree of science is important to reloading. However, at some undetermined point, the effects of your shooting art begin to outweight the effects of your reloading science (ie, your brand of primer probably doesn't matter if you can't hold quarter-MOA anyway). No one knows where this point is (and it varies for every shooter). I think, ultimately, you apply some basic statistical methods, try a couple brands of popular bullets, and find a load that you can shoot well out to whatever distance you like to shoot. This may not result in the "best possible ever" load for your rifle, but it doesn't take hours and hours of experimentation and number crunching either - hours which would likely be better spent at the range, perfecting the "artistic" side of your shooting.</div></div>
 
Re: Reloading Art vs Scientific Method

Hell, I can't even balance my checkbook. Don't know how I could comprehend anything sceintific about loading ammo.
 
Re: Reloading Art vs Scientific Method

I'm not necessarily looking for something that is going to magically make my gun shoot smaller groups - I already do pretty fair in that department. I'm more interested in it from the position of wanting loads that are more 'robust' - something like OCW (already use it), but with a bit more depth to it. If someone is happy with what they have, more power to 'em.
 
Re: Reloading Art vs Scientific Method

I have a decent background in math and science and have some earlier superficial acquaintance with DOE and have thought about it relating to reloading. But I have a few questions when you say you have the "best" load I wonder how that load works when the temp is 40 degrees different than when tested?

If you are not careful DOE may get you an optimized solution that is not robust to minor differences between shooting sessions. One thing the ladder test or OCW methodolgy can provide for us is to find the widest sweet spot. For example you may find a range of .4 grains of powder that gives you acceptable accuracy whereas another sweet spot may give you slightly better accuracy but with a far narrower range of powder weights. I will usually go with the wider sweet spot so I know it will shoot in a wider range of conditions.

I also tend to shoot calibers that are known to be accurate calibers. I think these calibers for whatever reason have wider sweet spots. Lately I have been shooting quite a bit of 6.5 x 47L, 6.5 x 55 AI, 6 Dasher, and 6 BR. Shooting these rounds allows me to develop very accurate loads rapidly and spend my other shooting at range to develop my other skills.

As an example of accurate calibers I was developing loads for a 6 BR I have and did a ladder test over a range of Varget that was 1.6 grains wide. I shot 27 rounds and only two of them were outside of the 10 ring when shot on an electrically scored target at 300 meters. With that kind of performance I could not say the 2 that were outside the 10 ring were my fault or the loads. I looked with the 27 rounds and found what I thought was a sweet spot loaded for it and shot it and shot great in any condition I shot it in. Was that the best load inside that 1.6 grains I have no idea as I am not sure that any load within that window would not have worked just as well.
 
Re: Reloading Art vs Scientific Method

wwbrown

good point
that is why I determined the range of powder load for my DOE with a ladder test too. The results told me more about primers and seating depth than powder load (margin means plot slopes were more striking in the former than the latter)
 
Re: Reloading Art vs Scientific Method

Great thread. I know I have a student version of JMP around here somewhere. I need to dig it up.
 
Re: Reloading Art vs Scientific Method

Guys, since group size response to charge weight is non-linear, how are you modeling that response? Quadratic? Cubic?
 
Re: Reloading Art vs Scientific Method

Good question!

Somewhere I had been reading about something along those lines... that with only two levels per factor you'd only be able to see linear effects; in order to see the higher order effects you'd need at least three levels see any curvature to that line, and at least four to see if it bends like an 'S'...

I think that was one of the 'problems' mentioned with some of the fractional factorial methods - that you might not see some of the higher order interactions, or be able to really analyze them. My understanding was that the recommendation was for two-level factorial or fractional factorial screening tests to identify the main effects, followed by more detailed full factorial tests involving more levels in order to better identify the interactions one is interested in.

I wish I could remember where I saw that at; I read thru enough material in the last week that it kind of started to blur together. Guess I should have been taking notes w/ references as I went
wink.gif
 
Re: Reloading Art vs Scientific Method

I am all about using quantitative methods, and I thought that DOE sounded great, but then I started thinking about the fact that experimental design typically uses linear functions. During load development, what we do is try to exploit non-linearity in the response variable (typically group size, though some methods use other response variables) with the charge weight or other stimulus variable (charge weight and seating depth being the most common).

I am trying to think of the best way to simultaneously optimize seating depth and charge weight. Most people find the best charge weight first, and then find the best seating depth at that charge weight. I have always found that the seating depth I tested with yields the best accuracy, which makes sense as seating depth and charge weight interact. Typically, best loads are around 100% load density with 308 and 175 SMK's and Varget powder.

The problem here is that the surface is a lot mor complicated than a surface plot might yield. And the second you start showing the charge-to-accuracy relationship as a quadratic, cubic, quartic, or higher polynomial, you end up having to shoot a lot more groups, at which point, you are back to square one.

What I AM interested in, though, is finding better ways to measure accuracy to take advantage of the many data points. It is inefficient to shoot 5 shots to yield one piece of data. CEP (circular error probable) and RSD (radial standard deviation) show a lot more promise.
 
Re: Reloading Art vs Scientific Method

Carter,

Have you made any attempts at using CEP, RSD, AGR, or any other alternate methods of group size measurement? I will admit that going to the lengths as described in that paper I linked earlier on CEP, with using targets specifically generated for the purpose and all... looks like a lot of work. I've wondered if OnTarget or similar products could give most of the benefits with a lot less hassle. I know the 'free' version of OnTarget will calculate Mean Radius; I seem to recall that RSI Shooting Lab would export the X-Y coordinates in a CSV type format that could potentially be used for other calculations, though its been 5+ years since I've used that package.

For those interested... a couple more links on this topic:

Statistical notes on rifle group patterns

Group Statistics
 
Re: Reloading Art vs Scientific Method

On Target uses "Average to Center," which is directly proportional to CEP using the Raleigh Estimation. In the past, this is what I have used for testing rimfire lots, as opposed to group size.

With On Target, you can use 1 bullet per target and use the point of aim function to get X,Y coordinates for each shot (with the group center being for a 1-shot group). After that, calculating CEP is cake. Unfortunately, if you have multiple bullet holes on one target, you can't extract the data on individual bullets (at least, not that I can figure out how to do).

I have always liked RSD (in theory), but it looks like CEP does much the same thing and we can actually use On Target to calculate it (by doing 1 shot groups as I have described).

In the past, I have shot 5 X 5's to select rimfire ammo. I don't like doing more than 5 shots per target because the holes can be hard to identify, so I get 5 pieces of data from 25 shots. This looks a little more laborious, but 25 shots will get me a very good number. The problem that I have now is wind shifts that will cause my X, Y coordinates to drift. This heavily penalizes lots that are shot over a shifting wind. I isolate that now by shooting 5-shot groups over a very brief time frame, usually less than 30 seconds per group.