Market Research Data Processing and Survey Tabulation

Data cleaning here means survey data cleaning. We are not selling a warehouse pipeline. The checks are the ones that change a crosstab: completes under the LOI floor, identical grid patterns, duplicate devices, answers that the skip logic says should be empty, and open ends that are gibberish or copied prompts.

You can give us the rules. Many agencies have a written speeder definition and a straight-line definition they need applied the same way on every wave. We follow that spec. Where you do not have one, we propose thresholds from the LOI distribution of the study and wait for a yes before we delete records.

Data tabulation is the banner plan: stubs, banners, nets, and the statistics under them. We build multi-level banners to your spec. Demographic cuts, nested variables, top-two-box, means, medians, standard errors, and NPS are routine. Significance testing is z-tests and t-tests at 90, 95, and 99 percent, marked on the table so a reader can see a difference without redoing the math in the margin.

A second person checks bases, nets, and the direction of the letters against the source margins. The failure mode in data tabulation is a right-looking table with a wrong base. That check is the job, not an extra.

Field rarely lands on the census or the customer profile you specified. Data weighting puts the achieved sample back toward that profile. For a single target, cell weighting is enough. For several at once, age, gender, region, and income being the usual set, we use iterative proportional fitting, which most clients call rim weighting.

Weighting survey data is also a design-effect problem. Heavy weights make a table look more precise than the interviews underneath it. We report the effective base. If a weight solution is extreme, we say so and show the unweighted result beside it rather than burying the tradeoff in a footnote nobody sees.

A practical limit, which we would rather say before the run than after a client panics: if one respondent would have to count as four or five people to hit a target, the target is wrong for the sample you fielded. The fix is a collapsed banner, a revised target, or more interviews. It is not a cleverer weight. We will recommend which of those three is honest.

Open ended questions do not tabulate until someone builds a codeframe. Our coders, working in the languages the study fielded, write a deductive frame from the questionnaire and an inductive frame from what people actually said. Both get a review before the full file is coded.

Data coding covers brand lists, “other specify,” and longer verbatims. Nets match the table spec, not the coder’s personal sense of tidiness. If the study is multi-country, coding can follow the translated verbatims from the localization desk so a theme in one market is the same code in the next.

Multi-response questions need the same discipline. A “select all that apply” brand list with an exclusive “none of these” has to be checked for people who ticked both. Nets of overlapping items have to be defined before the table is run, or the percentages will add past 100 in a way that looks like an error and sometimes is one. The net rule goes into the spec so the next wave does not invent a different one.

A tracker is where market research data processing either earns its fee or invents a story. The speeder rule, the codeframe, the weight targets, and the banner shells have to be the same ones used last wave, unless you have written down a change. An analyst who “improves” a net in wave four will show a jump that the market did not produce. We keep a processing note per wave for that reason. New codes wait in a labeled other-specify until you promote them.

Multi-market files have a second version of the same problem. Countries finish on different days, and someone always asks for an early read on the markets that closed. We will cut an interim table. We will also mark it as interim, with the bases, so a half-finished Germany cell is not briefed as the German result. When the last market closes, the full run replaces it. One study, one spec, no orphan spreadsheets named “final_v7”.

Deliverables: labeled SPSS (.sav), Quantum, WinCross, SAS, ASCII, and Excel, including pivot-ready sheets when the client’s analysts live there. Charts are available. The default is the table and the data file, because most of our clients would rather build their own slides than receive a deck that paraphrases their spec.

If the study was programmed by our desk, the variable names already match the tables. If it was programmed elsewhere, we build a map and send you the mismatches before anyone waits on a wrong net. Either way, the processing note lists the exclusions, the weight targets, and the banner version. Keep it with the file. Six months later it is the only way to remember why a base moved.