Bytehub

Appearance

Surveys ​

  • one of the most commonly used research techniques.
  • strength: large number of responses quickly.
  • respondents can be geographically dispersed.
  • can help capture the "big picture"

Survey Pros and Cons ​

Pros ​

  • easy to collect data from a large number of people.
  • do not require (advanced) tools for development.
  • can be distributed easily.

Cons ​

  • not very good at getting detailed data.
  • can lead to biased data. (recall bias: subject to interpretation or memory)
  • not possible to ask follow up questions.

population of interest == target population == targeted users (in hci)

the population of interest is filtered based on inclusion criteria such as profession, etc...

Probabilistic Sampling ​

in a probability sample it is known exactly how likely it is to select a participant from the sample.

Stratification ​

  • A stratified sample is when you divide your entire population in separate subpopulations, known as strata.
  • A separate sample is drawn within each subpopulation.
  • stratification can help to have subgroups of the same size.

Example:

  • Survey of BIT students:
  • Subpopulations: first-years, second-years, third-years
  • Each year has different number of students, but stratified sample invites the same number from each year.

Non-probabilistic Sampling ​

  • goal in HCI is not usually population estimates.
  • usually population is not well defined.

Non probabilistic sampling examples:

  • volunteer opt-in panels.

  • self-selected surveys.

  • snowball recruiting. (respondents recruit other people)

  • ask about demographic data to confirm validity.

oversampling: asking a large amount of people relative to the estimated population size, to avoid biases.

  • Random sampling of usage:

    • user is asked to fill a survey every 10th time they load a website.

  • Self selected survey:

    • respondent finds the survey online.
    • most natural data collection method.

Developing Survey Questions ​

  • open ended questions.
  • close ended questions
    • scale (excellent - poor)
    • yes or no questions.

Common problems ​

  • two questions in the same question.
  • use of negation
  • biased wording
  • hot button words

Overall Structure ​

  • instructions
  • nice layout
  • not too long

Online vs Paper ​

different people will have different access to both. So depends on the audience.

Testing ​

  1. review by colleagues and analysts
  2. Interviews with potential respondents.
  3. pilot study of the survey tool and implementation procedures.

How to increase response rate: ​

  • offer reward
  • introductory letter
  • ease of submitting
  • multiple contacts with respondent

Data analysis ​

  1. quantitative and qualitative data is separated
  2. data is cleaned: removed invalid responses.
  3. if it is quantitative data use descriptive statistics
  4. if it is qualitative data use content analysis (chapter 11)