Design of Experiment
The different designs available
WHAT IS DESIGN of EXPERIMENT?
A group of techniques (main effects, factorial, surface response, mixture, Taguchi, Shainin, …)which are used to improve our understanding of processes.
To have confidence (normally set at 95%) in the results of a trial. Not to find out later the results we achieved were freaks.
To identify the critical factors in a process.
To understand the relationship between inputs and outputs, so a formula/ model of a process can be built which predicts the output values from the input values.
To determine the optimal settings for the inputs.In my experience, the single biggest causes of ineffective trials are poor planning and an expectation by the inexperienced that the DoE technique is a magic wand to provide good results from little effort.
REMEMBER: No free lunches.
Senior managers need to understand the methodology of the technique so they can support engineers by asking the right questions, have the right vision and at the same time have realistic expectations.It seems to me, that we all agree PDCA (DMAIC) is the best way to carry out a trial, but then there is pressure to get a quick result which causes a short circuit in the brain.
THE THREE STEPS TO UNDERSTANDING A PROCESS
Usually, a DoE is a series of trials, each step leading you to the next, untyil you have a good understanding of the process.
Step 1. Main Effects Trial: To define the parameters most likely to be influencing he critical outputsIt is not unusual for there to be 50 or more parameters which are potentially influencing a process.
We need a technique to help us prioritize, to choose the most likely parameters that are influencing the output.
A useful set of main effect designs are those defined by Plackett and Burman. The advantage of these designs is that by ignoring interactions, it will be possible to investigate lots of main effects with a minimum number of runs.
There is high risk as we are ignoring interactions. Should there be a very strong interaction, it will appear as in the analysis as an apparent influential main effect; interactions are heavily correlated with each other and with the main effects. Nevertheless, it is a sensible first step to help eliminate non-influential parameters from future trials.
The P&B design (all orthogonal arrays) with 8 runs can investigate 7 parameters. 12 runs - 11 parameters: 16 runs - 15 parameters: 20 runs - 19 parameters: 24 runs - 23 parameters: 32 runs - 31 parameters.
An orthogonal array is one where no column is correlated with another. If you carry out a correlation study on an orthogonal array the correlation between columns is zero.
The orthogonal arrays designed by P&B are shown below.
I have only shown the 1st rows here.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
8 run + + + - + - -
12 run + + - + + + - - - + -
16 run + + + + - + - + + - - + - - -
20 run + + - - + + + + - + - + - - - - + + -
To define the 2nd row:the first level (+ or -) from row 1 is put at the end and all others move left one place.
To define the 3rd row:the first level from row 2 is put at the end and all others move left one place.… and so on, until
The last row are all negatives (-)
P&B also has designs for 4, 24, 32, 36, 44 and 48 run trials, not shown here as they are unlikely to be used in packaging.
Taguchi also has a Main Effect design, the L12 which is a 12 run array capable of investigating 11 factors.
… for its P&B designs, uses an inverted reflection of these plans;
i.e. the first line shown here, is reversed and becomes the penultimate line in Minitab
.… does not use the 8 run P&B.
Step 2. A Factorial Trial:To investigate interactions and create an initial model of the process.
The very origins and strength of DoE are in this technique. It was developed by RA Fisher in the early part of the 20th century for agricultural experiments. It is only when using a factorial can we investigate interactions; the classic one-at-a-time trial cannot detect interactions; they are the process knowledge that allows us to really control a process.
Lack of knowledge about interactions leads to one operator discovering that to increase the weight of the product it is necessary to increase pressure at the input, whilst a second operator discovers the reverse.
The limitations of a full factorial come from having only two levels per factor, thus the model can only be based on a first degree polynomial; hence only an estimation of the true model. The full factorial notation of Xk is X levels for each of k factors.The number of runs in a full factorial trial (all orthogonal arrays) where each parameter is tested at two levels will be 2k. For example a full factorial, 2 levels for 4 factors will be 24 = 2×2×2×2= 16 runs .. but a full factorial investigates all interactions.
Often we take the risk in packaging of only investigating second order interactions (e.g. pressure × time) not bothering about third order interactions (e.g. pressure × time × temperature) and above. In Minitab this is referred to as a resolution V design. This allows us to investigate more factors.
In the table below I show the designs to be avoided, those in the shaded area.
The Box-Behnken Design: is illustrated here as a three factor trial.
It does not have the classic corner points, but mid-points between the corners, plus the classic centre point at the middle of the process window. The advantage of the Box-Behnken design is that it avoids extremes of settings. Imagine a ball which just touched the corner points of the cube shown below. All the experimental points of the B-B design are well inside the ball.
Step 3. A Response Surface Trial: An improved second degree polynomial model of the process.
There are a couple of designs in common use (and included in Minitab’s software); the Central Composite and the Box-Behnken .
These designs use the process window of your interest, and identify suitable positions across the window to test. These will allow an improved model of the process. They provide good data for the model without the need of multiple levels and without loss of trial efficiency
The Central Composite Design: is illustrated to the right with a three factor trial. It is called a composite, because it has three composite elements:
1- a classic full factorial of 8 (red) corner points
2-the star element of 6 (green) points at a distance of alpha from the centre
3-a single (blue) centre point.
Alpha is defaulted to the square root of k, where k is the number of factors, but can be set to meet the demands of a special case; for example if by taking the trial outside the corner point cube, an input value becomes negative.
In addition to the classic three step approach (Main effects – Full factorial – Surface plot), there will be occasions when these designs do not address the requirements of a trial. I mention two subsets below.
A common problem can be likened to a recipe problem. What percentages of three different ingredients should I use in my recipe. Absolute values are not necessarily important, but the percentage composition is important.
The most common, illustrated here with a three factor trial. This shows how the design is made using points equally spaced points across the process window. .
When this type of percentage issue is found within a process with classic factors, then it is possible to design a trial which combines the classic factorial with the recipe approach.
Here there is a classic process with three factors (Bore, Angle and Contour), with the added factor of Colour which is a percentage issue of three colours (Red, Sienna and Teal).
The colour trial is carried out at four of the eight corner points.
Taguchi Designs: To create highly customized trials.
Learn about this topic in RK:QM course Trials level 2
Taguchi is a very controversial person, some love him, and some think his work is dangerous. One statistician refers to his work as worse than a broken leg. We cannot deny though, that he has had a very influential impact upon quality improvement, not least his loss function concept.
His contribution to designs for trials, from my personal experience, has also been very useful; on many occassions I have successfully sorted a problem with a very cost effective design.
Taguchi’s designs for factors which cannot be controlled, using inner and outer arrays, are just brilliant!
Always in packaging there is a concern about the cost of trials. Combine that with the need to frequently carry out a trial with four or five levels in a factor (different designs, different colours, …) and you have a conflict of interests. Especially if one of the factors takes two days to change over from one level to another.
To the despair of the purist statistician, we will take short cuts; but it is important to acknowledge the risk and stay very aware of what we are doing.
The techniques I use most often are linear graphs leading to customised designs.
This allows me to design a trial with, for example:
5 factors at 2 levels, (BRS, BRG, FRS, GR, VI)
1 factors at 3 levels (CS)
1 factors at 4 levels (FRP)
... combine this with our experience of where there are likely to be no interactions.
We can use this information to prepare a linear graph (as shown on the right, and then design the resulting 32 run trial.
There are huge savings in this approach, but it is dangerous in the wrong hands.
Not strictly a trial technique, it is more a technique to identify the critical process input parameters.
The Pareto principle (devised by Juran) identifies the 20% of factors that cause 80% of the variability. Shainin calls such critical factors Red X factors; we generally say the x axis of a chart is the input and the y axis the output.
A typical trial to decide if a factor is a Red X looks like this.
run A B C D E
1 - + + + +
2 + - - - -
If the results from these two runs are not significantly different, the factor A is a Red X (critical process factor).
Look also at Multi-Vari charts (developed by Len Seder)
Very infrequently do you find a simple trial design in packaging processes. Get help!