Choosy with words in Salary Surveys: split-ballot experiment - March 01, 2016

Writing good survey questions is complex. Careful decisions need to be made, in particular as regards the wording. Obviously a question can be asked in numerous ways. Each way however, may differ when it comes to being understood, e.g. some wordings may be more familiar than others. Previous studies have shown that unfamiliar terms can induce survey break-off and reduce response quality. How to deal with that problem?

One way to deal with the choice of words is to count how often particular words are used in books, newspapers, magazines and on the internet. WageIndicator used these frequencies to evaluate the Wageindicator questionnaire, both the English and Slovenian language version. We checked the frequency of all nouns, verbs, adjectives and adverbs and made a list of all words with a frequency in the lower 25%. For these we looked up synonyms and other possible alternatives, at the word and phrase level. Next we prepared a selection of wordings that could/should be improved and created an alternative questionnaire using these. Then, in Febraury 2016, we started a split-ballot experiment. Half of the respondents now receive the old questionnaire and the other half the 'improved' version. The results of the experiment will help us understand the relation between wording familiarity and response quality. Hopefully the experiment will contribute to (even more) clearly understood language in all WageIndicator questionnaires.

News every year