One of my hopefully sparse mantras is: ‘methodology is very important, but nothing like as important as we think it is.’ What do I mean by this? Well, here are a few common fallacies to kick off with:
- There is one right way: ok, few would sign up to the idea that there is a single correct way of conducting sociological research, that is, of answering a question or solving a problem. But it often seems otherwise. That the idea has any currency is perhaps mostly a by-product of blinkered Comte-like missions to render sociology a respectable, law-generating science (although quantitatively and qualitatively-oriented can alike display such missionary tendencies).
- We need to have proof: scientists cannot trade in proofs, although the illusion that they can persists. Popper rightly insisted on an acknowledgement of ‘fallibilism’ in science; and all scientific theories necessarily share this vulnerability. A moment of reflection on sociologists’ part on the fickleness of contingency and happenstance, and of the intrusion of agency too, should push the point home.
- It’s the best we can do: The best we can do does not necessarily comprise an optimal research strategy. Many possessing skepticism in relation to (1) and (2) might yet resort to ‘approximations’ of (1) and (2). An accessible date-set based on a probability sample of a national population can so seduce investigators that they become content only to ask questions answerable within the confines of the data-set. Many a neo-positivistic aspiration genuflects to (1) and (2).
- Follow the recipe: methods textbooks of all leanings, increasingly specialized and esoteric, frame and ‘shape’ studies. Following recipes faithfully and accurately, it is assumed, is a guarantee of valid, reliable and significant findings. If only the social world was this accessible to sociological taming!
- Satisfy peer review: allegiance to (4) often strikes as a precondition for publication in academic journals of merit, where the legitimacy of a piece of research is underwritten by reference to authoritative recipes. Mere mention of noted texts, even authors, is sometimes substituted for the adoption of recipes by aspirants for the reflected glory of an international journal’s impact factor.
- It’s flawed, chuck it out: there are flaws and flaws. This fallacy refers to those defined by the flawed ambitions of we sociologists. Those who reach beyond what can credibly be accomplished can be overly ready to throw up their hands in despair, and the baby out with the bathwater, if they detect a falling short of targets (1) to (5).
These fallacies lend themselves to a less than optimal return on investment in sociological research. But to wariness about this should be appended another concern, namely, the avoidance of crude, stereotypical attributions to quantitative versus qualitative methods, and to a binary-breaking ‘third way’ of mixed methods.
Silverman has listed the familiar objections to quantitative and qualitative studies. The former:
- amount to a ‘quick fix’, involving minimal contact with people or the ‘field’;
- yield statistical correlations based on variables that are arbitrarily defined;
- give rise to post hoc interpretations about the meanings of correlations;
- allow values to ‘seep into’ measurement decisions with complex and contested concepts, and
- rely on a purely statistical logic that trivializes hypothesis testing.
And of qualitative studies, they:
- have problems of consistency or reliability, given their focus on short data extracts;
- raise issues of validity;
- amount to little more than ‘anecdotalism’;
- sidestep the issue of representativeness, and
- do not lend themselves to hypothesis testing.
But is mixed methods research the answer? It would seem logically so: after all, one surely shouldn’t simply have to choose between numbers and words? Besides, interpretively-oriented qualitative work is necessarily involved in the selection and reduction of chunks of reality to variables; and positivistically-oriented quantitative work is necessarily evoked in insinuations of the representativeness of case studies. But not so fast.
Hammersley cautions against conflating the notion of ‘triangulation’ with combining quantitative and qualitative methods. Triangulation can involve different combinations of either quantitative or qualitative methods, rather than crossing the divide between the two. Moreover, the idea of mixed methods research ‘preserves the quantitative-qualitative division while seeking to bridge it’. He writes:
‘the problem is not that this distinction does not refer to significant variation in how researchers go about their work. To the contrary, the problem is that it refers to very many sorts of variation. And just as it is best to see triangulation operating at a more micro level than the combining of different broad approaches – for example, we should think of it in terms of using data from interviews with different people or observations in different settings, combining different forms of interviewing or observation, and so on – so too it is better to see the differences between qualitative and quantitative as operating at a more specific level. The qualitative-quantitative distinction can refer to variation in at least the following aspects of research problems (a more ‘inductive’ versus a more hypothetico-deductive approach), the planning of research (more emphasis on initial or recurrent planning), the collection of data (more versus less structured approaches), data analysis (use of counting, tables, statistical techniques versus reliance on qualitative and discourse analysis), and writing research reports (standard format versus flexible format, depending on what is being reported, what audience is being addressed, etc). Now the point is that there is no automatic link between most of the choices that are made about each of these aspects of the research process and the choices that are made about others. It is possible to combine a relatively inductive approach with using quantitative data; to collect unstructured data and then turn it into quantitative form; to report qualitative research in terms of standard format, and so on.’
In short, a positive approach to mixed methods undermines any tendency to assume that there are impermeable boundaries between the quantitative and qualitative. More negatively, an uncritical advocacy of mixed methods research can treat the distinction between quantitative and qualitative as if it were ‘more uniform, stable and meaningful than it really is.’
Pawson too warns against an unthinking displacement of the binary of quantitative and qualitative methods by a third orthodoxy, ‘mixed methods’. His discussion departs from Hammersley’s however. He contends that sociologists seem only belatedly to have discovered that society is ‘multi-faceted, multi-layered and multi-perspectival.’ This calls for a fusing of both disparate forms of available evidence and disparate paradigms and perspectives. He laments the emergence of a new breed of multi-methods technocrats who proffer spurious solutions that ‘always end up in witchcraft’ (the only product is a ‘mere hybridity of method’). Finally, he argues that what is needed is well-rounded and better grounded theory. The quandry about how to combine quantitative and qualitative research is solved the moment we begin to theorize about how ‘processes’ lead to ‘outcomes’. Mixed methods per se is no Holy Grail. Indeed!
I have in innumerable blogs expounded and commended – and tried to deploy and illustrate – retroduction and abduction via Bhaskar’s philosophy of critical realism. I will not reproduce these arguments here, but rather refer to Blaikie’s helpful ‘methodological’ explications.
Blaikie sets out the deployment of retroductive inference as follows:
- in order to explain observable phenomena and the regularities that obtain between them, scientists must attempt to discover appropriate structures and mechanisms;
- since structures and mechanisms will typically be unavailable to observation, it is necessary to first construct a model of them, often drawing on already familiar sources;
- a model is such that, were it to represent correctly those structures and mechanisms, the phenomena would then be causally explained;
- the model is then tested as a hypothetical description of actually existing entities and their relations. To do so, it is necessary to work out further consequences of the model (that is, additional to the phenomena to be explained), that can be stated in a manner open to empirical testing;
- if these tests are successful, this gives good reason to believe in the existence of these structures and mechanisms;
- it may be possible to obtain more direct confirmation of these existential claims, by the development and use of suitable instruments;
- the whole process of model-building may then be repeated, in order to explain the structures and mechanisms already discovered.
Pawson and Tilley cite Durkheim on suicide in this context:
‘Durkheim, whilst accepting that the decision to commit suicide was, of course, a matter of the individual’s misery, depression, isolation and so on, was able to show that such dispositions are socially structured and thus vary with the social cohesion and social support which different communities, localities, organizations, family groupings and so on are able to bring to marginalized members. He demonstrated that the supremely ‘individual’ act of suicide is in fact ‘socially’ structured and so produced the famous research cataloguing differences in suicide rates: higher in (individualistic) Protestant communities than in (collectivist) Catholic communities; higher for widowers in their competitive (male) networks than widows in their communal (private) female networks; higher for the childless in their (restricted) family roles than for parents with their (extended) family ties; higher on (insouciant) weekends than (well-structured) weekdays; and so on.’
So, individual choices are constrained or enabled by an individual’s contextual circumstances. Thus both agency and structures enter the explanation. Explanation in the retroductive approach is achieved by establishing the existence of the structure or mechanism that produces the observed regularity. Prediction is not possible given the open nature of social systems.
Abduction refers to the inference from social actors’ accounts to those of sociologists (that is, from Schutz’ lay ‘first order’ to sociologists’ ‘second order’ typifications). It is historically associated with the interpretivist opposition to positivism and critical rationalism. Drawing on Schutz, whom he cites throughout (forgive the sexist terminology), Blaikie spells out an abductive research strategy as follows:
- the social scientist observes certain facts and events within social reality which refer to human action;
- ‘he constructs typical behaviour or course-of-action patterns from what he has observed’;
- ‘he co-ordinates to these typical course-of-action patterns a personal type, a model of an actor whom he imagines as being gifted with consciousness’;
- ‘he thus ascribes to this fictitious consciousness a set of typical notions, purposes, goals (in-order-to motives corresponding to the goals of the observed course-of-action patterns and typical because-motives upon which the in-order-to motives are founded), which are assumed to be invariant in the mind of the imaginary actor-model’;
- the puppet ‘is invested with a system of relevances originating in the scientific problem of his constructor and not in the particular biographically determined situation of the actor within the world’;
- the puppet ‘is interrelated in interaction patterns to other … puppets constructed in a similar way’;
- this constitutes a model of the social world of everyday life;
- the system of typical constructs designed by the social scientist must be formed in accordance with the following postulates:
- the postulate of ‘logical consistency’ – they must supercede first-level constructs by conforming to the principles of formal logic and by being clear and distinct;
- the postulate of ‘subjective interpretation’ – it must be possible to refer ‘all kinds of human action or their result to the subjective meaning such action or result of an action had for the actor’;
- the postulate of ‘adequacy’ – ‘a human act performed within the lifeworld by an individual actor in the way indicated by the typical construct would be understandable for the actor himself as well as for his fellow-men in terms of commonsense interpretations of everyday life’
Thus, in much the same way as social actors test first-level constructs, ideal types can be verified by predicting ‘how a puppet or system of puppets might behave under certain conditions’. In this way propositions can be developed which state relations between sets of variables. For a different, if compatible, articulation of the role of abduction in research, see Chris Yuill’s excellent article on alienation in Social Theory and Health.
Personally, I would argue that a retroductive, rather than inductive or deductive, quantitative or variable-oriented research strategy can often be effectively and advantageously combined with an abductive qualitative or case-oriented research strategy. Lay or first-order narratives on the one hand, and regularities (‘regs’) and partial regularities (‘demi-regs’) on the other, each afford clues and cues as to what is happening and why in chosen corners of interest to us in our open system.
This blog is a precis of an earlier paper:
Scambler,G (2010) Qualitative and quantitative methodologies in comparative research: an integrated approach? Salute e Societa IX 2/2010 Supplement.
Other References
Blaikie,N (2000) Designing Social Research: The Logic of Anticipation. Cambridge; Polity Press.
Durkheim,E (1951) Suicide. New York; Free Press.
Hammersley,M (2008) Troubles with triangulation. In Ed Bergman,M: Advances in Mixed Methods Research. London; Sage.
Pawson,R (2008) Method, mix, technical hex, theory fix. In Ed Bergman.M: Ibid.
Pawson,R & Tilley,N (1997) Realistic Evaluation. London; Sage.
Silverman,D (1998) Qualitative/quantitative. In Ed Jenks,C: Sociological Dichotomies. London; Sage.