Exercise 3.4 asks you to consider the value of questionnaire surveys in the light of the previous discussion of alternative research strategies and approaches.
Questionnaires and interviews, the techniques which are at the heart of one type of survey research, occupy a major place in small-scale social science research projects. This is the case to such an extent that, when students or employees are asked to carry out a research study, they almost automatically think of using these techniques, often both at once. Interviews and question-naires can also, of course, play a part in action research, case studies and experimental approaches to social science research. Yet they are not the only techniques available, with the use of documents and observations also widespread.
from questions of method [and tools], and give further consideration to the purposes and research questions, bearing in mind that the way questions are asked influences what needs to be done to answer them.
(Punch 2005: 240) As this quote reminds us, you need to consider some of the broader issues of research design and philosophy before focusing on the tools and techniques for data collection. Box 3.16 poses some further questions to help you revise, reflect upon and reformulate your plans. In addition, there are further issues which you may need or wish to consider to help you reach your decision. Eight are considered in this section.
What do you need or want to find out?
One of the key determinants of the approach you might use in your research is undeniably the nature of the research proposed. You may want, or need, to answer a particular question or set of questions. This may immediately suggest a method or technique to you. For example, if you want to find out what members of the general public think about a given issue, an obvious way forward is to ask some of them. Or, if you need to understand why a traffic management plan does not appear to have reduced traffic congestion, observing traffic behaviour in the relevant area may seem appropriate.
Box 3.16 Which method to choose?
1 Research Questions: What exactly are you trying to find out? Focus on the
‘exactly’ as this can lead you either into the quantitative or qualitative direction.
2 Are you interested in making standardized and systematic comparisons or do you really want to study this phenomenon or situation in detail?
3 The Literature: How have other researchers dealt with this topic? To what extent do you wish to align your own research with standard approaches to the topic?
4 Practical Considerations: Issues of time, money, availability of samples and data, familiarity with the subject under study, access to situations, gaining co-operation.
5 Knowledge payoff: Will you learn more about this topic using quantitative or qualitative forms of research? Which approach will produce more useful knowledge? Which will do more good?
6 Style: Some people prefer one approach to another. This may involve paradigm and philosophical issues or different images about what a good piece of research looks like.
(Source: adapted from Punch 2005: 239–40)
You might want to have another look at Box 2.3 at this point.
In the research literature, it is sometimes suggested that if your research questions are well enough focused or refined, they will effectively determine the methods you use to answer them. In practice, however, in almost every case, there will be alternative techniques which could be used, either instead of or in conjunction with the one(s) you first think of. Thus, to follow up the two examples just given, you might observe the public’s actual behaviour, where this impacts upon the issue under consideration, or you might attempt to model traffic behaviour using a computer programme.
The words ‘want’ and ‘need’ in the title of this sub-section also suggest an important distinction, one which has to do with focusing as well as method.
Here, as when considering your research plans in general, you need to think about which methods are practicable given the time and other resources you have available.
What skills do you have?
As we suggested in the opening section of this chapter, on Everyday research skills, one of the key resources you have available for your research is yourself.
You will have many skills as an adult which have been developed through everyday life. So you might find it useful to think about what skills you are best at and what skills you like using.
Do you like talking to people? How comfortable are you with the give and take of conversation? Do you like watching people? Would you prefer to sit at a desk and read documents? Or work on data stored on a computer? Each of these preferences suggests that you might be most comfortable using a particu-lar technique: some form of interviews, observation or documentary analysis.
You might, therefore, find it easiest, where possible, to research an area which allows you to use the skills you have best developed, or which you are most comfortable with. Use Exercise 3.5 to help you decide.
Alternatively, you might, in carrying out an audit of your everyday and research skills (see Exercise 1.1), decide that you want to use your research project to deliberately develop skills which you feel you lack, but which could be useful to you. This might be a good way of adding interest or personal benefit to a research project which could otherwise be rather boring or limit-ing. Be careful, however, not to over-reach yourself if this might affect the likelihood of you successfully completing your project.
Will your methodological preferences answer your questions?
Bearing the two previous questions in mind, you now need to consider whether the everyday research skills which you wish to use are actually suited to the issues you are going to investigate.
To return to the two examples used above, if you like talking to people, and feel that you are reasonably good at it, you will probably be comfortable with the idea of researching public opinion by talking to people. Or, if you are experienced with computer modelling, you may be engaged by the prospect of researching traffic management options using these techniques. If the circumstances were reversed, however, you might have some qualms.
There may, as has already been suggested, be ways around such problems, short of changing your research topic. In the latter case, if computers turn you off and you’d much rather talk to people, you might research traffic manage-ment by questioning a number of drivers and pedestrians, perhaps referring to existing computer-based studies as necessary. Or, in the former case, you might approach the study of public opinion by re-examining some of the many sources of published information, rather than by questioning people directly.
In many cases, however, particularly where you do not have an entirely free choice over the subject of your research, you will find yourself needing to use methods you may not feel entirely happy with.
How will your methods affect the answers you get?
Just as it is sometimes argued that your research questions should determine your approach and techniques, so, in an analogous fashion, it is often sug-gested that the methods you use will significantly affect the answers you get.
There is, of course, a good deal of sense in this argument.
If you carry out a questionnaire survey, the information you collect will be rather different from what you would get if you used in-depth interviews, though there might be a good deal of overlap. The questions, and thus the nature and scope of the answers, in a questionnaire are determined in advance by the researcher. Interviews, by contrast, even when highly structured, allow for more flexibility in asking and answering questions. While the responses to the former tend to be brief, those to the latter may be very lengthy, so fewer interviews may be carried out in the same time.
It is also the case, of course, that the choice of the subjects or objects of your research – people, classes, traffic, books, etc. – assuming that you are not able to study every possible subject or object of relevance to your questions, will influ-ence your findings. This issue is discussed further in the section on Sampling and selection in Chapter 6.
How will you affect your research?
Y soon realized that in fieldwork interactions, her nationality was often subordinate to her social position. The respondents perceived her as someone from ‘abroad’ first, and as an Indian later. She also realized that her native tongue (Hindi) did not allow her to pass as an ‘insider’. In her experiences, many of the respondents attempted to create a ‘good
impression’ by constructing their responses in broken English despite her repeated desire to conduct the interviews in their native dialects. The dilemma for Y then was whether to work with incomplete responses or to make a concerted effort to ‘de-glamorize’ her location. These problems also meant that Y chose not to reveal to her respondents that she was engaged to a white, European man because she feared that she would be understood by the respondents as an illegitimate ‘outsider’.
(Thapar-Bjorkert and Henry 2004: 367–8) Researchers are themselves a powerful, and often under-recognized, influence on their research and their findings. This influence extends beyond the choice of the research topic itself, and the methods used to explore it, to the impact of you as a person and of your ideas. As a researcher, you will have certain opin-ions and views about a wide range of issues, and these are likely to find some expression in your research and your reporting of it. Thus, as well as having a set of research questions to ask, you may already have a view on the likely answers. Your views may have been shaped by what you have read. This will affect the way in which you ask questions, of people or of documents, and the significance you attach to their answers. It will also affect who or what you ask, using your contacts or networks, and the ways in which your questions are answered.
As an adult, you also have a range of individual demographic characteristics, including your sex, age, class, ethnic background and size. These will impact upon your research in more or less complex ways, and will raise issues related to the contexts of your research. For example, if you are a white, middle-class male, conducting fieldwork in a women’s refuge, or in inner city areas with large black populations, will require you to consider the ways in which your sex, class and ethnicity contribute to your research findings. While such clear differences between the researcher and the researched throw these issues into relief, they are relevant in other cases as well.
If somebody else carried out your research, using the same approach, tech-niques and sample, the results would be, at the very least subtly, different.
There is no easy way in which the effect of the researcher on the research can be minimized. You cannot be wholly objective, and, in many ways, it is foolish to try to be so. The play of emotions between researcher, researched and research is often something to be welcomed and celebrated. Yet there is a need to be aware of your influence on your research, and to be as open as you can in recording and recognizing these affects. Box 3.17 lists ten questions to help develop your reflexivity.
Which methods are acceptable?
Another key issue in deciding about your approach to your research is the question of which methods may be acceptable. You may be working under direction or in collaboration with others, for example, and these people may
have an influence on your choice of methods. Your research may be funded by an organization which has very definite views on the value of alternative tech-niques, or has produced a research brief which limits or directs your choice. Or you may be working within a subject or disciplinary tradition which expects you to take a given methodological perspective.
In all of these cases, however, it should be useful to you to be able to reach a considered opinion on the advantages and disadvantages of using particular approaches or techniques. You may also be able to go a little further, and modify or add to the choice of methods.
Using more than one method
It is common, of course, for researchers to use more than one method. This is even more likely if you are carrying out your research project as part of a group, rather than on your own. Your main method may, for example, be a question-naire survey, or a set of interviews, or a series of observations, but this is likely to be complemented, at the very least, by some documentary analysis to enable you to explore relevant literature or policy. Most research projects in the social sciences are, therefore, in a general sense, multi-method.
There are, however, good reasons for deliberately seeking to use more than one method in the main body of your research. You may follow up a survey with some interviews, in order to get a more detailed perspective on some of the issues raised. The telling anecdote may be much more revealing and influential than almost any amount of figures. You might follow the reverse process, using interviews in order to identify the key issues which you would
Box 3.17 Developing reflexivity: some questions to ask yourself
• Think about a situation you have been in, preferably in piloting your research.
• What was your role in this situation?
• Did you feel comfortable or uncomfortable? Why?
• What actions did you take? How did you and others react?
• Was it appropriate? How could you have improved the situation for yourself, and others?
• What could you change in the future?
• Do you feel as if you’ve learned anything new about yourself or your research?
• Has it changed your way of thinking in any way?
• What knowledge, from theories, practices and other aspects of your own and others’ research, can you apply to this situation?
• What broader issues – for example, ethical, political or social – arise from this situation?
• Have you recorded your thoughts in your research diary?
Box 3.18 Eleven ways to combine qualitative and quantitative research 1 Logic of triangulation. The findings from one type of study can be
checked against the findings deriving from the other type. For example, the results of a qualitative investigation might be checked against a quantitative study . . .
2 Qualitative research facilitates quantitative research. Qualitative research may: help to provide background information on context and subjects; act as a source of hypotheses; and aid scale construction.
3 Quantitative research facilitates qualitative research. Usually, this means quantitative research helping with the choice of subjects for a qualitative investigation.
4 Quantitative and qualitative research are combined in order to provide a general picture. Quantitative research may be employed to plug the gaps in a qualitative study that arise because, for example, the researcher cannot be in more than one place at any one time. Alternatively, it may be that not all issues are amenable solely to a quantitative investigation or solely to a qualitative one.
5 Structure and process. Quantitative research is especially efficient at getting at the ‘structural’ features of social life, while qualitative studies are usually stronger in terms of ‘processual’ aspects . . .
6 Researchers’ and subjects’ perspectives. Quantitative research is usu-ally driven by the researcher’s concerns, whereas qualitative research takes the subject’s perspective as the point of departure . . .
7 Problem of generality. The addition of some quantitative evidence may help [generalizability] . . .
8 Qualitative research may facilitate the interpretation of relationships between variables. Quantitative research readily allows the researcher to establish relationships among variables, but is often weak when it comes to exploring the reasons for those relationships. A qualitative study can be used to help explain the factors underlying the broad relationships . . .
9 Relationship between macro and micro levels. Employing both quantita-tive and qualitaquantita-tive research may provide a means of bridging the macro–micro gulf. Quantitative research can often tap large-scale, struc-tural features of social life, while qualitative research tends to address small-scale behavioural aspects . . .
10 Stage in the research process. Quantitative and qualitative research may be appropriate to different stages of a longitudinal study.
11 Hybrids. [W]hen qualitative research is conducted within a quasi-experimental (i.e. quantitative) research design.
(Source: adapted from Punch 2005: 241–2)
then ask questions about in your survey. You might complement interviews within an institution with the analysis of available documents, in order to compare written and spoken versions.
Where two or more methods are used in this way, to try and verify the validity of the information being collected, the process is referred to as tri-angulation. This kind of approach should be carefully considered if your resources allow. Box 3.18 (previous page) suggests eleven ways in which quali-tative and quantiquali-tative research approaches may be productively combined, while Box 3.19 gives two examples of research projects where mixed methods have been applied.
Box 3.19 Two examples of mixed method research
In order to explore and collect data concerning facets likely to lead to teacher satisfaction or dissatisfaction in schools a focus group of seven teachers was constituted. The focus group included teachers from both the primary and secondary sectors, at varying levels of seniority within their careers, placed in an environment where they felt comfortable with self-disclosure. Teacher discussion within the focus group phase yielded forty possible facets which encompassed organizational values, organiza-tional climate, managerial processes, teaching and learning and self-realization . . . All forty facets . . . were included in a survey instrument created in the form of a questionnaire to be distributed to schools . . . together with an explanatory letter.
(Source: Rhodes et al. 2004: 69) The choice to undertake qualitative interviews in the context of a case study seemed an obvious one given our objective of listening to working people and asking about their experiences as learners. Given the enor-mous range of occupations and work settings in the public sector, the survey was seen as essential to provide a wider profile of the workforce.
The case studies involved semi-structured interviews with corporate managers and trainers and union branch representatives. This was com-plemented by more intensive observation of particular departments . . . the case studies in effect became ethnographies of a number of depart-ments or units. The survey was conducted in one local authority and one health service trust. It was a self-completion postal survey which aimed to gain a picture of the educational and competence profile of the workforce, including details of recent employment-related training and educational participation. Four departments were chosen from each organization for the survey on the basis of having a large proportion of
‘unskilled’ workers.
(Source: Munro et al. 2004: 292–3)