TRADE-OFF STUDY SAMPLE SIZE: HOW LOW CAN WE GO?

By Dick McCullough

Download This Article

Table 4.

Sample Bias Study 4

Finally, the synthetic data study below involves more closely replicating the study design used in this paper.

Method

The general approach was:

• Generate three data sets
• Each data set consists of utility weights for three attributes
• Utility weights for the first and third data sets are randomly drawn integers between 1 and 20
• Sample size for the first data set is always 200
• Sample size for the second and third data sets varies across 25, 50 and 100
• The second and third data sets always are of the same size
• The second data set consists of the first n cases of the first data set, where n = 25, 50 or 100
• Define either a two, three, four or five product scenario
• Estimate logit-based share of preference models for each of the three data sets, calculating shares at the individual level, then averaging
• Calculate MAEs for each of the second and third data sets, compared to the first, at the aggregate level
• Calculate MPEs (mean percent error = (1- rsq(utils-first data set, utils-other data set))*100) for each of the second and third data sets, compared to the first, at the aggregate level
• Redraw the sample 50 times for each scenario/sample size and make the above calculations
• Calculate mean MAEs and MPEs for each of 50 random draws for each model
• 36 models (3 data sets x 4 market scenarios x 3 sample sizes)

Note: Empirically, the ratio of random sample MAE to overlapping sample MAE equals the scalar that corrects the overlapping sample MAE for sample bias.  Similarly for MPE.  The issue, then, is to develop a formula for the correction factor that closely resembles the ratio of random sample error/overlapping sample error.

Previous Page | Next Page

MACRO CONSULTING
Intelligent Decisions