UCLA Labor Center

Survey Design

The design of your survey is central to your study, as the design supports the goals of your project (what you want to learn), and is specific to your sample (whom you will interview).  The design of your survey is tailored  to your methodology (how you will interview) and is also the final product of your pilot questionnaire (test run of your survey).

To better understand this process, here’s how survey design works with the aforementioned components in mind:

Table 2

Table4

Citation for table here

This section will review the initial process of designing your questionnaire and the general components of survey design.  If you would like to review in-depth each aspect of the survey study design, click here to download a survey design tutorial.  You can also download the Survey Fundamentals manual here to review non-technical language that synthesizes the underlying principals of a good survey design and implementation. Information for this page has been adapted from a variety of sources cited at the bottom of this page.  

Note: your survey is not a survey until it has been finalized and sent to your participants, in the meantime it is referred to as a questionnaire.  It may seem like we use these terms interchangeably, but the terms are utilized according to the phase of the study that is being described.  


Survey Vocabulary

Before we dive into the actual process of designing your survey, lets review vocabulary that is oftentimes utilized when constructing survey research:

Survey Questions

Survey questions can be open-ended (allowing the participants to explain their response) and/or close-ended questions (limiting the participants response choices to the answer choices in the questionnaire).  Click here to see examples of open-ended and close-ended survey questions and understand the strengths and limitations of each type of question.

Level of Measurement

The types of survey questions you chose to implement in your questionnaire will determine the measurement you can employ.  For example:

  • Dichotomous questions- are by definition a question that has two possible responses (i.e., yes, no and true, false).  Dichotomous variables are variables that have only two distinct valid values.  When you are analyzing your data, dichotomous responses become dichotomous variables (click here to learn more about variables if this transition is unclear).  Dichotomous variables are a specific type of metric variables that usually result in a proportion or a percentage. 

  • Multichotomous (aka Multiple Choice)- This question type displays a list of choices that are defined by the survey. Respondents can select more than one answer or choice, the response can be nominal (no order) or ordinal (a clear order).  In regards to measurement, these are referred to categorical variables.  Categorical variables take a value that is one of several possible categories. As naturally measured, categorical variables have no numerical meaning but a categorical variable can be coded to look like a quantitative variable by assigning numbers to categories (i.e., gender, race, age, etc.) . 

  • Ranking (also ordinal)- a ranking question asks survey participants to compare a list to one another and employ an order to the list.  For example, “Please rank each of the following statements to your experience with #1 being the most true and #10 being the most inaccurate.  A natural ordering exists for these categories and in advanced statistics these ordinal variables are treated differently to account for the structure of the variable.  The best way to determine central tendency on a set of ordinal data is to use the mode or median; the mean cannot be defined from an ordinal set.

  • Likert Scale- A “Likert scale” is the sum of responses to several Likert items that are part of a scale.  A “Likert item” is a statement that the respondent is asked to evaluate. Likert scales can be utilized to construct survey questions that attempt to measure on an interval level.  Click here to access sample Likert scales and Likert items.  Interval measures the distance between attributes (something that is not done with ordinal data) to compute an average of an interval variable.  

Population and Sample

You’re probably wondering how many participants need to take the survey (sample) in order to make sure that it accurately represents the population (entire set of people you want to understand) you are studying.  In order to arrive to this number, you need to know: 1) the size of your population, 2) how accurate your data needs to be, 3) the margin of error that you’re operating under; 4) the responses that you estimate you will receive, and 5) how many people you will need to send your survey to.   Not to worry, you can access the sample size calculator here.

Types of Surveys

Surveys can be divided into two broad categories: a questionnaire and the interview. Questionnaires are usually paper-and-pencil instruments that the respondent completes. Interviews are completed by the interviewer based on the respondent says. Note that closed and open-ended questions can be part of either a questionnaire or interview. Below is a list of commonly utilized surveys and their respective advantages and disadvantages:

Types of Surveys
 

Type of Survey Advantages Disadvantages
Face-to-face Interview Explain questions, explore issues, Expensive, need
make observations, use visual aids. interviewer training
– at home or work Accuracy, better sampling Expensive
– in public places Cheaper, more people in less time Less representative sample
Telephone survey Accurate, cheap No personal observations
Written questionnaire Cheapest per respondent Bias from low response rate
– by mail Allows anonymity Slow
– by e-mail Cheaper, quicker results Less representative sample
– web survey Quicker data processing Need computing expertise

citation here for this table


Design Basics

Now that you’re familiar with the survey vocabulary, lets review the basics of designing the survey.   From understanding when a survey is appropriate to utilize to managing the length and quality of your survey, the video below demystifies the process:

Watch this short clip to better understand the three main types of survey design and review additional survey best practices.

[youtube id=”10nMNh3RMp0″]

Survey Research in Action

At Keio University, Professor Isamu Yamamoto, in the Faculty of Business and Commerce, is conducting research with the aim of designing appropriate systems for the labor market. To do this, Professor Yamamoto analyzes how human resources should be allocated to take into account diverse values among businesses and workers, and the relationship between work-life balance and workers’ mental health and business performance.

[youtube id=”Fli6AcxbbkM”]

Click here to download the corresponding research paper, “How are Hours Worked and Wages affected by Labor Regulations?: The White-collar Exemption and ‘Name-only Managers’ in Japan”

citation here

This paper examines whether overtime regulations have a significant impact on the hours worked and hourly wages, by focusing on so-called name-only managers in Japan. The term name-only manager refers to an employee who has essentially the same job description as other employees, but who is designated by the company as a manager to exempt them from overtime regulations. As the name implies, the only difference between those managers and other employees is in the applicable regulations on working time. Using longitudinal data, our main results from matching estimation indicate no significant difference in hourly wage or hours worked between the two groups. This implies that name-only managers’ base salaries are sufficiently higher to compensate for the loss of overtime pay, which supports the fixed-job model.