Independent Direct-Mail Test Office
Independent direct-mail measurement
One decision. One protocol. One answer you can defend.
Compare two direct-mail vendors through a neutral, blinded test built around your brand's outcomes and economics.
- Brand-paid
- Vendor-neutral
- Targeting IP stays private
Illustrated explainer with synthetic narration modeled from Alex Sofronas's authorized voice recording.
Two polished reports can still produce the wrong comparison
Different audiences, identity rules, attribution windows, exclusions, and conversion definitions can make a vendor “winner” meaningless. Vendors should not have to reveal targeting IP to one another, either.
Separate data
Each vendor submits its targeting file to a separate secure intake. Cross-vendor row-level data is never shared.
Shared rules
The decision, outcome, power threshold, attribution window, exclusions, and stopping rule are fixed before results are visible.
Brand-owned truth
The analysis uses outcomes for the full eligible population, not competing vendor matchback reports.
How the independent test office works
We design the comparison, protect the parties, and deliver a decision package that marketing, finance, procurement, and both vendors can inspect.
Frame the decision
Define the eligible universe, primary conversion, economic metric, minimum detectable effect, allocation, and decision rule.
Normalize and assign
Validate joins, suppressions, overlap, duplicates, and match coverage, then randomize before outcome access.
Analyze blind
Measure overlap performance, unique-audience incrementality, uncertainty, and fixed-budget policy value.
What you receive
- Signed test charter and decision question
- Power and minimum-volume memo
- Secure intake schema and vendor instructions
- Immutable randomization and balance record
- Audience waterfall from eligibility through analysis
- Lift, 95% confidence intervals, and economics
- Winner, no clear winner, or inconclusive ruling
- Executive report and reproducible technical appendix
Who qualifies
A full head-to-head is usually a fit when both vendors will follow one charter, the brand can provide outcomes for the full eligible universe, and a targeting-only test can hold creative, offer, format, timing, and frequency constant.
Meaningful decision
Often $750,000+ in annual direct-mail spend or comparable contribution is at stake.
Sufficient volume
Often 37,000–66,000 analyzable households per arm, or enough prespecified drops to pool.
Reliable identity
At least 85% of intended records can usually be joined to stable household IDs.
These are planning guides, not automatic exclusions. Smaller or more complex programs can begin with a power review. Vendor-attributed matchback reports alone qualify for a measurement audit, not an independent winner-selection test.
Choose the right starting point
Readiness & Power Audit
$7,500–$10,000
Two weeks to assess data readiness, identity and overlap feasibility, power, risks, budget, and timing.
Test Blueprint
$15,000–$20,000
A signed charter, schema, cell design, sample-size plan, randomization specification, and analysis plan for internal execution.
Independent Head-to-Head
From $45,000
Complete design, secure intake, normalization, assignment, quality control, blind analysis, fact check, and final reports.
Annual test office programs are typically $135,000–$225,000. Print, postage, creative, vendor media fees, identity services, and exceptional clean-room charges are client-paid or pass-through costs.
Independence is part of the product
The brand pays the core evaluation fee. We do not accept success fees, winner fees, media commissions, or undisclosed referrals. Conflicts are disclosed before signing, and vendors retain their targeting IP.
Do vendors see each other's lists or scores?
No. Each vendor uses a separate intake. Only aggregate overlap and quality findings are returned.
Do you guarantee a winner?
No. The preregistered method can produce a winner, no clear winner, or an inconclusive result. The rules do not change after outcomes appear.
Do you need a no-mail control?
Yes when the decision requires an incremental-value claim. The charter and power calculation determine the allocation.
How long does a full test take?
Typically 10–16 weeks, depending on readiness, mail execution, outcome timing, and review requirements.
Make the comparison fair before the mail drops
Start with a readiness and power audit. You will know whether a defensible test is possible, what data is missing, and the minimum volume required before committing to the full engagement.
Book the readiness conversation