Use the proper sample size: Always use a large enough sample size that will lead to statistically significant results – either by estimating performance on the front end or by ensuring the test results are valid on the back end. Try Avalon’s Sample Size Calculator for some quick guidance.

Create a control (or placebo) group: Always include a selected group that receives the proven treatment and/or package to give you a baseline standard against which you can compare your testing results. In the case of list testing in acquisition, this means you test the tried-and-true continuation lists against test lists that you’ve never mailed. The performance of your continuation lists makes up your baseline metrics.

Test one variable at a time: Make sure that your idea is the only thing (other than chance) that can affect your results. For example, if you want to measure the difference between a membership card and an invoice, all of the other components must be EXACTLY the same. Otherwise, your test is not a reply device test, but a test of two entirely different packages with lots of variables – which tells you nothing when it comes time to roll out.

Test for big impact – and managed risk: Focus your limited budget and time on the tests that will have the most impact on your program— messages, your offer, asking strategies, themes, segments or lists, and carrier and reply device strategies.

Balance your need to test against the reality of your financial restrictions – in other words, manage your risk. For example, you may not want to include lapsed donors in a full-file test of first-class postage, which would significantly drive up your costs. Instead, you could run a postage test among mid-range and high-value donors. Be sure to include roll-out costs in your decision making.

Code properly – and thoroughly – to ensure clear results:

• Establish separate source codes for each type of mailing or contact.
• Source coding schemes should isolate and measure all of the key variables and all elements you want to analyze (i.e., HPC, first gift date, donor type, package, etc).
• Package codes should be an isolated character in the source code; this will help clarify production instructions and avoid errors.
• Do not collapse segments you need to measure separately.
• Avoid frequent changes in source code logic – think it through the first time! Frequent logic changes make it difficult for long-term analysis.

Continue reading here for the 5 other essential elements of direct marketing testing.

Click here for our Top Six Testing Tips.