What are the steps in gathering information through survey?

Quality Glossary Definition: Survey

Table of Contents Show

When to Use Surveys to Collect Data
How to Administer a Survey
Survey Considerations
Survey Example

Variations: questionnaire, e-survey, telephone interview, face-to-face interview, focus group

Survey is defined as the act of examining a process or questioning a selected sample of individuals to obtain data about a service, product, or process. Data collection surveys collect information from a targeted group of people about their opinions, behavior, or knowledge. Common types of example surveys are written questionnaires, face-to-face or telephone interviews, focus groups, and electronic (e-mail or website) surveys.

Surveys are a valuable data collection and analysis tool that are commonly used with key stakeholders, especially customers and employees, to discover needs or assess satisfaction.

When to Use Surveys to Collect Data

It is helpful to use surveys when:

Identifying customer requirements or preferences
Assessing customer or employee satisfaction, such as identifying or prioritizing problems to address
Evaluating proposed changes
Assessing whether a change was successful
Monitoring changes in customer or employee satisfaction over time

How to Administer a Survey

Determine what you want to learn from the survey and how you will use the results.
Determine who should be surveyed by identifying the population group. If they are too large to permit surveying everyone, decide how to obtain a sample. Decide what demographic information is needed to analyze and understand the results.
Determine the most appropriate type of survey.

Determining Survey Type Example
Determine whether the survey’s answers will be numerical rating, numerical ranking, yes-no, multiple choice or open-ended, or a mixture.
Brainstorm questions and, for multiple choice, the list of possible answers. Keep in mind what you want to learn, and how you will use the results. Narrow down the list of questions to the absolute minimum that you must have in order to learn what you need to know.
Print the questionnaire or interviewer's question list.
Test the survey with a small group. Collect feedback.
Test the process of tabulating and analyzing the results. Is it easy? Do you have all the data you need?
Revise the survey based on test results.
Administer the survey.
Tabulate and analyze the data. Decide how you will follow through. Report results and plans to everyone involved. If a sample was involved, also report and explain the margin of error and confidence level.

Survey Considerations

Conducting a survey creates expectations for change in those asked to answer it. Do not administer a survey if action will not, or cannot, be taken as a result.
Satisfaction surveys should be compared to objective indicators of satisfaction, such as buying patterns for customers or attendance for employees, and to objective measures of performance, such as warranty data in manufacturing or re-admission rates in hospitals. If survey results do not correlate with the other measures, work to understand whether the survey is unreliable or whether perceptions are being modified by the organization’s actions.
Surveys of customer and employee satisfaction should be ongoing processes rather than one-time events.
Get help from a research organization in preparing, administering, and analyzing major surveys, especially large ones or those whose results will determine significant decisions or expenditures.

Survey Example

Pearl River School District, recipient of the Malcolm Baldrige National Quality Award, surveys students, parents, and staff annually using a questionnaire developed and administered by a national research firm. This ensures statistically correct methods, allows them to watch changes over time, and enables comparison with other districts. The surveys show areas of satisfaction and dissatisfaction, but not reasons. To find causes, they must dig deeper. Recently, the annual survey showed that the high school students ranked the cafeteria service low. This was hardly surprising, since the research firm’s database showed that almost all high school students rate their cafeterias low. However, Pearl River wanted to improve this rating.

A committee of student leaders and cafeteria staff prepared a second survey to get more information. Figure 1 below shows a condensed version of their survey. Demographic information was requested at the top. A short paragraph stated the reason for the survey, some additional information respondents needed to know, and who was eligible to complete the survey. The survey was designed to be filled out quickly with check marks in appropriate boxes. A few open-ended questions were placed at the end. Data from the survey, supplemented with sales data, was analyzed and used to plan menu changes.

Figure 1: Data Collection Survey Example

Adapted from The Quality Toolbox, ASQ Quality Press.

THE SET-UP STAGE OF A SURVEY

Survey Set-up

The set-up stage is a very important part of any research, yet its value is often underestimated. Most research projects (and therefore, most surveys) which go badly wrong, do so because of inadequate attention given to setting the project up properly. For this reason, any effort to collect information, should not occur until the issues discussed in this Section are dealt with. This Section is intended to assist researchers and managers in identifying the key factors in planning the research project, analyse their requirements and select the most appropriate method of collecting the necessary information.

Aims and Objectives

The first and most important step, in the collection of any information, is to establish the research objectives clearly. From this basis, it is then possible to translate these objectives into establishing specific information requirements. The objectives should define what the survey results will be used for and may include such uses as the basis for decision-making, the allocation of funds, to analyse the outcome of policies or programs (program evaluation) or to determine the direction of future operations. It is highly recommended that the number of objectives be kept to a minimum. With too many, the information gathering exercise can easily become too unwieldy and costly.

The Target Population

Once our uses for the data have been determined, our next requirement is to define the target population. The target population is the group about which we wish to make inferences from the survey data. The target population should be defined in terms of :

content (eg. all persons),
units (eg. in households),
extent (eg. in Australia), and
time frame (eg. 6 September 1991).

An examination of the examples listed in brackets above reveals the target population of the 1991 Census of Population and Housing. A clear definition, such as that shown above, will give the researcher some idea of the study's potential size, as well as some insight into possible survey methodologies. Note that the target population may not be easily accessible due to logistical or financial restraints. For example, in the Census of Population and Housing, it is difficult to obtain data on the homeless or people who are in transit from one place to another during Census day.

When considering the population that is to be studied, an important factor is the units that are to respond to the collection (the reporting units) and the units that are to be selected to access the reporting units (the selection units). An example of the difference between reporting units and selection units is given in Aims and Objectives, where the selection unit is a household and the reporting unit is any responsible adult resident at that household.

Methods of accessing populations are dealt with further in Frames & Population.

Conducting Preliminary Research

This step involves becoming familiar with the issues to be researched. By doing so, researchers obtain a stronger focus for their study, as well as some insights on how best to collect the desired information. Preliminary research is done by gathering together relevant reports, information papers, academic journals, books, newspapers and government files. Similarly, talking to people who are familiar with the issues should also be undertaken.

Establishing Data Requirements

The first consideration when clarifying information requirements is to clearly define the data items to be collected. Only with clear definitions of the terms and concepts involved can the researcher be assured that the data will be relevant to the project's aims and objectives. Standard definitions for data items should be investigated as the use of standards allows comparability between the results from the current investigation and previous data. A common way to ensure all the appropriate data items are collected is to specify, as early as possible in the planning stages of the collection, the detailed tables to be derived. Such specifications not only help clarify the data items to be collected, but the levels or subgroups for which the data is required. Assuming an appropriate collection methodology, defining the levels of interest beforehand will help to ensure an adequate sample size is chosen to achieve the desired levels of accuracy. The definition of concepts also plays a part in identifying the population. The survey objectives define the target population, but the specific data items and output requirements determine the make-up of subgroups inside the population. These subgroups make up the cells on the tables that have been determined in describing the output requirements. All data items proposed for the survey should be scrutinised closely to establish whether they are consistent with the aims and objectives of the project. It is vital that extra information is not sought because it is of interest or is related to the issues being examined. This may lead to the survey losing its focus and add considerably to the cost and time to do the project. Ideally the information requirements, whether they be factual data (eg. unemployment or inflation rates) or attitudinal data (eg. opinions on a particular topic), and the proposed methods of analysis should be put down in writing, as this can help to focus the researcher's attention on important considerations affecting the design of the collection.

Financial and Time Constraints and Other Considerations

Apart from the important considerations in survey design outlined above, there are several other aspects that need to be accounted for. The financial and time constraints placed on a survey are often a deciding factor influencing survey design, as well as the size of the survey and the level of accuracy sought. It is therefore important to make both time and financial estimates for each aspect of a survey and monitor them closely. Other aspects include creating or maintaining the survey frame, designing the sample selection, choosing a data collection method, designing the questionnaire and determining the field and data processing procedures.

NON-SURVEY METHODOLOGIES

As described above, the characteristics of our data items often point at the best way to collect our information. How expensive and time-consuming the data are to collect are also often important considerations when choosing whether to run a survey. The availability of similar data from other sources also plays a part, but collating data from many sources may also prove time-consuming and expensive.

Existing Data Sources

The information that is required may already be available from another data source. Research of subject matter documentation, as discussed in Conducting Preliminary Research above, may provide the necessary information on some topics. Some data may also be available from government departments and agencies as a by-product of their administrative information (eg. Registrar of Births, Deaths and Marriages). Previous surveys may also have been carried out in the field of interest. Care must be taken that the recording of the data comes from reliable sources and that the definitions of the data items used closely approximate those currently of interest. This is a cheap and fast way of obtaining data so long as the existing data meet the requirements of the research.

Case Study

In preparing a case study the researcher seeks to collect and analyse as much data about the chosen subject as possible from a relatively small number of cases. Case studies allow greater detail to be gathered for less cost than in most surveys, but the data is less likely to reflect the whole population.

Participatory Observation

Participant observation is a method of data collection whereby the researcher seeks to become a member of the group under study, and records data from within. This method is often used in qualitative studies ie. studies that deal with variables which cannot be readily measured such as peoples perception, group dynamics, etc.

Advantages and Disadvantages

The main advantages of non-survey methodologies is that they provide data, often quite detailed, in-depth data, at a relatively low cost and that they can result in substantial reductions in the time taken to produce results. However, statistical accuracy may be traded off to achieve these gains. Existing data sources may use definitions that do not match the required definitions very well, and the other non-survey methodologies use only very small numbers of subjects in the study and are open to the subjects not being representative of the target population. This does not mean that non-survey methodologies should be overlooked entirely - it only means that care should be taken when analysing their results. For example, if we chose to look at the effect of overcrowding in our hospitals, we may decide to perform a case study or control group experiment rather than a survey as these methods allow the researcher to collect information for a comparatively modest cost.

SURVEY DESIGN CONSIDERATIONS

After considering all the options available to you, should you decide that a survey is the option most suited to your study, your next task will be to decide upon the survey design. The three objectives of sample surveys are to

use the data collected from a sample of population units (using the sample statistics calculated),
to describe a population, i.e. estimate certain population parameters (descriptive statistics), and
to test statistical hypotheses about the population.

Issues relating to the survey design encompass all the methodological and organisational aspects of the survey to be conducted and involve making decisions about how these will be carried out. In doing this, decisions that are technically desirable need to be balanced off against what is practically feasible, and need to take into account the purposes of the survey, the accuracy required in the results, the resources available (ie financial, timing, staffing, and other resources) and other practical considerations. The aim of survey design is to best meet the aims of the survey within the boundaries of the resources that are available. Survey design issues to be considered include :

the survey objectives,
the population of interest (or target population),
the reference period for the data,
Geographic, demographic boundaries,
the frame and the units,
the sample design and sample size,
the data collection method,
the questionnaire,
non-sampling errors,
non-response,
the field procedures and data processing system, and
the tabulations and analyses required.

Some of these points have been dealt with in this Chapter. The rest are dealt with in the remaining Chapters.

SURVEY MANAGEMENT

Survey management involves organising and controlling each aspect of the survey. The staff responsible for the survey need to be skilled and experienced in order to avoid the many pitfalls that confront the researcher new to the field. The operational aspects of a survey require thorough planning and efficient management of financial and human resources. In the course of addressing survey management issues, a researcher may find that the survey needs to be modified before it can be conducted feasibly. Since survey management is an integral part of a survey, sufficient resources should be allocated to it to ensure that the survey runs smoothly and produces reliable and timely results.

Advertising

There are several parts of the survey where advertising is necessary. These include advertising for staff which can be done through the daily newspapers or through Centrelink and pre-survey publicity to potential respondents which can be done through an introductory letter sent to respondents. After the survey, the availability of survey results can be advertised through the media or by contacting interested parties. Publicity of the implementation of survey results can also raise the usefulness of the study. When advertising for staff, you should specify what the job is, subject of the survey, amount of interviewing work, need for prior experience if required, rate of pay, number of days interviewer is required for, whether evening/weekend work is involved and area where interviews will be conducted.

Finance

The financial constraints placed on a survey is often a deciding factor influencing the survey. It is therefore important to make financial estimates for each aspect of a survey and to monitor spending (and therefore the progress of the survey) closely. Adequate resources should be allocated to survey management as it plays a central role in the conduct of a survey, particularly the most labour intensive part. Financial estimates for other aspects of a survey generally fall into three categories:

overheads,
salaries, and
survey and processing costs.

Some areas requiring financial estimates includes advertising, hire of rooms, power, office equipment and supplies, printing of forms and questionnaires, postage, salaries of office and field staff, survey testing, travel expenses, data processing and publication and dissemination. All of these estimates need to be made at an early stage in the development of the study and also monitored along the way.

Recruitment

Regardless of particular collection methods used, personnel need to be recruited and trained to collect, despatch and process the survey questionnaires as well as interviewing respondents. When recruiting staff, it is necessary that they have the basic skills and attributes required to do the work, and in the case of a national survey, they must be in or near the areas where they are expected to work. The number of staff that you need to recruit depends on the size of the sample, the length and complexity of questionnaire, the timetable and the availability of funds. Staffing levels may vary during the research period depending on staff retention rates or whether there are peak periods or low periods.

Training

The adequacy of training given to interviewers and processing staff has a strong influence on the quality of results obtained from the survey. Thorough training of interviewers is important because they have a wide range of tasks to perform and are the main point of contact between the respondents and the researcher. Comprehensive training of office staff should enable them to process the survey questionnaires as accurately and as quickly as possible. Training can be provided in the form of:

manuals,
formal training courses, and
"on the job" training.

It is also useful to include some form of home study exercise to test understanding of instructions and survey procedures. Interviewers and processing staff will improve as they gain practical experience "on the job" and consolidate their formal training.

Interviewer Training

Training for interviewers includes the purpose of the survey; the scope and coverage of the survey; a general outline of the adopted sampling approach; the format of the questionnaire; recording of the responses; correct interviewing techniques; techniques for avoiding or reducing non-response and maintaining respondent co-operation; field practice; quality control; editing; planning the workload; and administrative arrangements.

Training for Processing Staff

For processing staff the training should cover the purpose of the survey; the scope and coverage of the survey; the questionnaire; the way the responses are recorded; coding; editing; key-punching instructions if the survey responses are to be punched onto a computer database; quality control; and administrative arrangements.