Glossary of Mixed Methods Terms/Concepts
The following terms are adopted from Tashakkori and Teddlie's (2003, Handbook of mixed methods in the social and behavioral research) Glossary. Please refer to the original for more details and for further references. Cited references with a date of 2003 are chapters in that Handbook.
Complementary inference
Conceptual (or Inferential)
Consistency
Conceptual Framework
Concurrent Mixed Method Design
Concurrent Mixed Model Design
Concurrent Nested Design
Concurrent Triangulation Design
Contrast Principle
Convergent Inference
Conversion Mixed Model Design
Data Consolidation
Data Conversion/Transformation
Data Quality
Deductive inference (in
research cycle)
Deductive logic
Design quality
Dialectical position
Divergent inference
Ecological transferability
Ecological validity (or ecological external
validity)
Effect size
External validity
Fully integrated mixed model
design
Fundamental principle of
mixed methods research
Generalizability
Gestalt principle
Inductive
inference (in research cycle)
Inference
Inference quality
Inference transferability
Interactive model
Internal validity
Interpretive agreement (or interpretive
consistency)
Interpretive distinctiveness
Measurement validity
Meta-inference (or integrated mixed inference)
Meta-interpretation
Methodological triangulated
design
Mixed Method Design
Mixed methods sampling
Mixed Model Design
Monomethod
design
Monostrand design
Multilevel mixed methods design
Multilevel mixed model design
Multilevel mixed sampling
Multimethods design
Multimethods QUAL
study
Multimethods QUAN
study
Multiple methods design
Multistrands design
Multitrait-multimethod
matrix
Operational transferability
Paradigm
Parallel mixed model design
Population transferability
Population validity (or population external
validity)
Qualitizing
Reliability (data reliability or measurement
reliability): See data quality.
Rules of integration
Sequential explanatory design
Sequential exploratory design
Sequential mixed method design
Sequential mixed model design
Similarity-contrast principle
Single phase design
Stage (of a study)
Strand
Supervenience theory
Temporal transferability
Temporal validity (or temporal external validity):
See inference transferability.
Transferability
Transformative-emancipatory perspective
Transformative mixed methods design
Triangulation
Two Phase Design
Validity (measurement validity or data
validity): See data quality.
Validity (design validity or inferential validity):
See inference quality.
Within-Design consistency
Complementary inference: This is when the results of two strands of a mixed methods study provide two different but non-conflicting conclusions or interpretations. Back to the top
Conceptual (or Inferential) Consistency: This refers to the degree to which the inferences are consistent with each other and with the known state of knowledge and theory. Back to the top
Conceptual Framework: This is a consistent and comprehensive theoretical framework emerging from an inductive integration of previous literature, theories, and other pertinent information. Conceptual framework is usually the basis for reframing the research questions and for formulating hypotheses or making informal tentative predictions about the possible outcome of the study. Back to the top
Concurrent Mixed Method Design: This is a multistrand design in which both QUAL and QUAN data are collected and analyzed to answer a single type of research question (either QUAL or QUAN). The final inferences are based on both data analysis results. The two types of data are collected independently at the same time or with a time lag. Back to the top
Concurrent Mixed Model Design: This is a multistrand mixed design in which there are two relatively independent strands/phases: one with QUAL questions and data collection and analysis techniques and the other with QUAN questions and data collection and analysis techniques. The inferences made on the basis of the results of each strand are pulled together to form meta-inferences at the end of the study. See also rules of integration. Back to the top
Concurrent Nested Design: This
is a concurrent mixed model design classified on the basis of (conceptual or
paradigmatic) dominance or priority of the study. In this design, a
quantitative strand/phase is embedded within a predominantly qualitative study (quan + QUAL) or vice versa (QUAN + qual).
QUAL and QUAN approaches are used to “confirm, cross-validate, or corroborate
findings within a single study” (Creswell,
Concurrent Triangulation Design: This is a concurrent mixed model design classified on the basis of purpose of the study. In this design, QUAL and QUAN approaches are used to “confirm, cross-validate, or corroborate findings within a single study” (Creswell et al., 2003). Back to the top
Contrast Principle: See similarity-contrast principle. Back to the top
Convergent Inference: This is when the conclusions or interpretations of two strands of a mixed methods study are consistent with each other (i.e., agree with each other). Back to the top
Conversion Mixed Model Design: This is a multistrand concurrent design in which mixing of QUAL and QUAN approaches occurs in all components/stages, with data transformed (qualitized or quantitized) and analyzed both qualitatively and quantitatively. Back to the top
Data Consolidation: This means combining qualitative and quantitative data to create new or consolidated variables or data sets. Back to the top
Data Conversion/Transformation: Collected quantitative data types are converted into narratives that can be analyzed qualitatively (i.e., qualitized), and/or qualitative data types are converted into numerical codes that can be statistically analyzed (i.e., quantitized). Back to the top
Data Quality: This is the degree to which the collected data (results of measurement or observation) meet the standards of quality to be considered valid (trustworthy) and reliable (dependable). This term has been used by Punch (1998) to represent “quality control of data (p. 257)” “in terms of procedures in the collection of the data, and … in terms of three technical aspects of quality of the data: reliability, validity, and reactivity, p.257)” (1) Data/Measurement validity: Do the results of data collection truly represent the construct or phenomenon that they are expected to capture (measure or represent)? “How well the data represent the phenomena for which they stand (Punch, 1998, p.258).” See also convergent validity and discriminant validity. (2) Data/Measurement reliability: Do the obtained results of measurement or observation accurately reflect the magnitude, intensity, or quality of the attribute or phenomenon that is being measured or observed? Back to the top
Deductive inference (in research cycle): This is a process in which hypotheses or predictions are formed on the basis of (1) a conceptual framework that is constructed from the literature, (2) the inferences of a previous strand of a mixed methods study, or (3) an existing theory. See also inference and inference quality. Back to the top
Deductive logic: (Erzberger & Kelle, 2003) (1) This refers to the application of general rules to specific cases. For example, from the general rule that all men are mortal, it can be deduced that if Socrates is a man, then he will be mortal. (2) This refers to a type of reasoning usually applied if a link is drawn from an already formulated theoretical statement to a statement about observable empirical facts, a link that can be generalized in the following term: “If A (a theoretical statement) is true, then we would expect the fact C to happen.” Back to the top
Design quality: See inference quality. Back to the top
Dialectical position: (1) (Greene & Caracelli, 2003) To think dialectically is to invite the juxtaposition of opposed or contradictory ideas, to interact with the tensions invoked by these contesting arguments, or to engage in the play of ideas. The arguments and ideas that are engaged in this dialectic stance emanate from the assumptions that constitute philosophical paradigms—assumptions about the social world, social knowledge, and the purpose of science in society. Back to the top
Divergent inference: (Erzberger & Kelle, 2003) This is when the inferences made on the basis of the two
strands of a mixed methods study are inconsistent or dissonant (Rossman &
Ecological transferability: This refers to generalizability or applicability of inferences obtained in a study to other settings or contexts. Subumes the QUAN term ecological validity and ecological external validity, and the QUAL term transferability See inference transferability. Back to the top
Ecological validity (or ecological external validity): See inference transferability. Back to the top
Effect size: This refers to the intensity, magnitude, or practical significance of an obtained result (e.g., relationship, difference) in the QUAL or QUAN strands of a mixed methods study. Onwuegbuzie and Teddlie (2003) explicitly relate this historically QUAN term to QUAL research, naming several new terms, including manifest effect size, frequency (manifest) effect size, and intensity (manifest) effect size. Back to the top
External validity: This is defined by Cook and Campbell (1979, p. 37) as follows: “the approximate validity with which we can infer that the presumed causal relationship can be generalized to and across alternate measures of the cause and effect and across different types of persons, settings, and times” (p. 37). See inference transferability. Back to the top
Fully integrated mixed model design: This is a multistrand concurrent design in which mixing of QUAL and QUAN approaches occurs in an interactive (i.e., dynamic, reciprocal, interdependent, iterative) manner at all stages of the study. At each stage (e.g., in formulating questions), one approach (e.g., QUAL) affects the formulation of the other (e.g., QUAN). See also interactive model. Back to the top
Fundamental principle of mixed methods research: Johnson and Turner (2003) define this principle as follows: “Methods should be mixed in a way that has complementary strengths and nonoverlapping weaknesses. … It involves the recognition that all methods have their limitations as well as their strengths. The fundamental principle is followed for at least three reasons: (a) to obtain convergence or corroboration of findings, (b) to eliminate or minimize key plausible alternative explanations for conclusions drawn from the research data, and (c) to elucidate the divergent aspects of a phenomenon. The fundamental principle can be applied to all stages or components of the research process.” Back to the top
Generalizability: See external validity and inference transferability. Back to the top
Gestalt principle: This refers to the whole or the totality. Gestalt psychology is known for the principle (among many others) stating that the whole is bigger than the sum of its parts. The Gestalt principle is applied to mixed methods ... to demonstrate that global inferences made at the end of mixed methods studies are more than the simple sum of the inferences gleaned from QUAL and QUAN strands. Back to the top
Inductive inference (in research cycle): This refers to a process of creating meaningful and consistent explanations, understandings, conceptual frameworks, and/or theories by integrating (a) the current knowledge gleaned from the literature, (b) concrete observations or facts, (c) results of data analysis in a research project, and (d) findings of a previous strand of a mixed methods study (Tashakkori & Teddlie, 1998). Back to the top
Inference: (1) This is an umbrella term referring to a final outcome of a study. The outcome may consist of a conclusion about, an understanding of, or an explanation for an event, a behavior, a relationship, or a case. (2) This is “a conclusion reached” where there is either (a) a “deduction from premises that are accepted as true” or (b) an induction by “deriving a conclusion from factual statements taken as evidence for the conclusion” (Angeles, 1981, p. 133). See also deductive inference (in research cycle), deductive logic, inductive inference (in research cycle), inductive logic, meta-inference (or integrated mixed inference), and retroductive inference. Back to the top
Inference quality: (1) This is proposed as a mixed methods term to incorporate the QUAN term internal validity and the QUAL terms trustworthiness and credibility of interpretations (Tashakkori & Teddlie, 1998, 2003). The definition of the term is as follows: the degree to which the interpretations and conclusions made on the basis of the results meet the professional standards of rigor, trustworthiness, and acceptability as well as the degree to which alternative plausible explanations for the obtained results can be ruled out. Inference quality consists of Design Quality (within-design consistency) and Interpretive Rigor [conceptual (or inferential) consistency, interpretive agreement (or interpretive consistency), and interpretive distinctiveness]. Back to the top
Inference transferability: This refers to generalizability or applicability of inferences obtained in a study to other individuals or entities (see population transferability), other settings or situations(see ecological transferability), other time periods (see temporal transferability), or other methods of observation/measurement (see operational transferability). It subsumes the QUAN terms external validity and generalizability as well as the QUAL term transferability. Back to the top
Interactive model: (Maxwell & Loomis, 2003) Applied to mixed
methods research, this model indicates that “the different components of actual
mixed methods studies are … connected in a network or web rather than a linear
or cyclic sequence.” Back
to the top
Internal validity: See inference quality. Back to the top
Interpretive agreement (or interpretive consistency): This refers to consistency of interpretations across people (e.g., consistency among scholars, consistency with participants’ construction of reality). Back to the top
Interpretive distinctiveness: This is the degree to which the inferences are distinctively different from (and superior to) other possible interpretations of the results and the rival explanations are ruled out (eliminated). Back to the top
Measurement validity: See data quality. Back to the top
Meta-inference (or integrated mixed inference): This is an inference developed through an integration of the inferences that are obtained on the basis of QUAL and QUAN strands of a mixed methods study. Back to the top
Meta-interpretation: Forthofer (2003) describes this term as follows: “Mixed methods designs are inherently more complex, and those that attempt any integration or synthesis of results across methodologies require an additional phase of “meta-interpretation.” See also inference. Back to the top
Methodological triangulated design: (Morse, 2003) This is a project that is comprised of two or more subprojects, each of which exhibits methodological integrity. While complete in themselves, these projects fit to complement or enable the attainment of the overall programmatic research goals. Back to the top
Mixed Method Design: (1) This is a design that includes both QUAL and QUAN data collection and analysis in parallel form (concurrent mixed method design, in which two types of data are collected and analyzed), in sequential form (sequential mixed method design, in which one type of data provides a basis for collection of another type of data), or where the data are converted (qualitized or quantitized) and analyzed again (conversion mixed method design). (2) (Bazeley, 2003) This design includes studies that “use mixed data (numerical and text) and alternative tools (statistics and text analysis) but apply the same method, for example, in developing a grounded theory.” See also Mixed Model Design. Back to the top
Mixed methods sampling: (Kemper, Stringfield, & Teddlie, 2003). This is a simultaneous selection of units of study through both probability (to increase generalizability/transferability) and purposive sampling strategies (to increase inference quality). Back to the top
Mixed model design: This is a design in which mixing of QUAL and QUAN approaches occurs in all stages of the study (formulation of research questions, data collection procedures and research method, and interpretation of the results to make final inferences) or across stages of the study (e.g., QUAL questions, QUAN data). In multistrand designs, either the strands are parallel (concurrent mixed model design) or sequential (sequential mixed model design, in which inferences of one strand lead to questions of the next strand) or the data are converted and analyzed again to answer different questions (conversion mixed model design). Back to the top
Monomethod design: See monostrand design. Back to the top
Monostrand design: These designs use a single research method or data collection technique (QUAN or QUAN) and corresponding data analysis procedures to answer research questions. They are also known as single-phase designs. Back to the top
Multilevel mixed methods design: This is a design in which QUAL data are collected at one level (e.g., child), and QUAN data are collected at another level (e.g., family) in a concurrent or sequential manner to answer different aspects of the same research question. Both types of data are analyzed accordingly, and the results are used to make inferences. Because the questions and inferences all are in one approach (QUAL or QUAN), this is a predominantly QUAL or QUAN study with some added components. In practice, because research questions and the inferences that are made at the end of the study are usually both QUAL and QUAN (using mixed models), this design is not common. See also multilevel mixed model design. Back to the top
Multilevel mixed model design: This is a design in which QUAL data are collected at one level (e.g., child) and QUAN data are collected at another level (e.g., family) in a concurrent or sequential manner to answer interrelated research questions with multiple approaches (QUAL and QUAN). Both types of data are analyzed accordingly, and the results are used to make multiple types of inferences (QUAL and QUAN) that are pulled together at the end of the study in the form of “global inferences.” See also multilevel mixed method design. Back to the top
Multilevel mixed sampling: (Kemper et al., 2003). This is sampling strategy in which probability and purposive sampling techniques are used at different levels of the study (e.g., student, class, school, district). Back to the top
Multimethods design: This refers to designs in which the research questions are answered by using two data collection procedures or two research methods, both with either the QUAL or QUAN approach. See also multimethods QUAL study and multimethods QUAN study. Back to the top
Multimethods QUAL study: This refers to designs in which the research questions are answered by using two QUAL data collection procedures or two QUAL research methods. Back to the top
Multimethods QUAN study: This refers to designs in which the research questions are answered by using two QUAN data collection procedures or two QUAN research methods. Back to the top
Multiple methods design: (Brewer & Hunter, 2003) This refers to designs in which more than one research method or data collection and analysis technique is used to answer research questions. They include mixed methods designs (QUAL + QUAN) and multimethods designs (QUAN + QUAN or QUAL + QUAL). Back to the top
Multistrands design: This refers to designs that use more than one research method or data collection procedure. See also multimethods design. Back to the top
Multitrait-multimethod matrix: This is a matrix of correlations between multiple methods of measuring each of a set of attributes. The diagonal values indicate the reliability of each measure/method. The off-diagonal values indicate the convergent validity and discriminant validity of each procedure/instrument. This method was introduced by Campbell and Fiske (1959) to evaluate the quality of data obtained from measurement instruments. Back to the top
Operational transferability: This is the degree to which the inferences that are made on the basis of the results of the study are generalizable to other methods of observing/measuring the entities or attributes that the inference is about. Subsumes the QUAN terms external validity of operations and operational external validity (see Ary, Jaccobs, and Razavieh, 2003). Back to the top
Paradigm: (1) (Mertens, 2003). A conceptual model of a person’s worldview, complete with the assumptions that are associated with that view. (2) (Caracelli and Green, 2003) paradigms are social constructions, historically and culturally embedded discourse practices, and therefore neither inviolate nor unchanging. Back to the top
Parallel mixed model design: See concurrent mixed model design. Back to the top
Population transferability: This refers to generalizability or applicability of inferences obtained in a study to other individuals or entities. Subumes the QUAN term population validity and population external validity, and the QUAL term transferability See inference transferability. Back to the top
Population validity (or population external validity): See inference transferability. Back to the top
Qualitizing: This is the process by which quantitative data are transformed into data that can be analyzed qualitatively. Back to the top
Reliability (data reliability or measurement reliability): See data quality. Back to the top
Rules of integration: (Erzberger & Kelle, 2003) A set of rules can be formulated that may be helpful for drawing inferences from the results of qualitative and quantitative studies in mixed methods designs. These rules should be understood as general guidelines whose significance vary according to the research question, the empirical domain under investigation, and the specific methods employed. Erzberger and Kelle list eight rules of integration. See also inference. Back to the top
Sequential explanatory
design: According to Creswell et al.(2003), this
design “is characterized by the collection and analysis of quantitative data
followed by the collection and analysis of qualitative data. Priority is
typically given to the quantitative data, and the two methods are integrated
during the interpretation phase of the study.” Back to the top
Sequential exploratory design: According to Creswell, et al. (2003), this design “is characterized by an initial phase of qualitative data collection and analysis, followed by a phase of quantitative data collection and analysis. Therefore, the priority is given to the qualitative aspects of the study. Back to the top
Sequential Mixed Method Design:
(Onwuegbuzie and Teddlie, Chapter 13, this volume.) A design in which one type
of data (e.g. QUAN) provides a basis for the collection of another type of data
(e.g. QUAL). It answers one type of question (QUAL or QUAN) by collecting
and analyzing two types of data (QUAL and QUAN). Inferences are based on
the analysis of both types of data. This term subsumes “sequential study,
“two‑phase design ,” “sequential QUAL-QUAN Analysis” and “sequential
QUAN-QUAL analysis”. Back to the top
Sequential Mixed Model
Design: A multi-strand mixed (QUAL-QUAN, or
QUAN-QUAL) design in which the conclusions that are made on the basis of
the results of the first strand (e.g. a QUAN phase) lead to formulation of
questions, data collection, and data analysis for the next strand (e.g. a QUAL
phase). The final inferences are based on the results of both strands of
the study. The second strand/phase of the study is conducted to
either confirm/disconfirm the inferences of the first strand, or to provide
further explanation for unexpected findings of the first strand. Back to the top
Similarity-Contrast principle: A principle underlying data analysis in both qualitative and quantitative research. Governs the process of grouping of units of analysis (e.g., statements, items of a test) into categories (e.g., themes, factors, components, clusters) that are similar to each other, and distinctly different from other groups of such units. Also see Convergent Validity and Discriminant Validity. Back to the top
Single Phase Design: A study with either a qualitative phase or a quantitative phase. See Monostrand Design. Back to the top
Stage (of a study): A step or component of a strand/phase of a mixed methods study, i.e. conceptualization, method, inference. Back to the top
Strand: A phase of a mixed
methods study in which a QUAL or a QUAN approach is used in the method of
study, in data collection procedures, or in data analysis. Phases/stands
might be concurrent (parallel, simultaneous) or sequential, or they might include
conversion of one type of data to another for analysis (conversion). See
also Monostrand Design and Multistrand Design. Back to the top
Supervenience theory: This
is a theory that holds that certain facts or properties exist and can be explained
by a type of “dependency” relationship they have to each other. Thus, social
facts “depend” (or supervene) on the existence of individuals but cannot be
completely explained by this relation. Back to the top
Temporal
transferability: This refers to generalizability
or applicability of inferences obtained in a study to other time periods. See Inference Transferability. Back to the top
Temporal validity (or temporal
external validity): See Inference Transferability. Back to the top
Transferability: This term was used by Lincoln and Guba (1985, p.300)
as a qualitative analogue to external validity. See Inference Transferability. Back to the top
Transformative-emancipatory
perspective: (Mertens, Chapter 5, this volume) Mertens described
this perspective in the following way: “Transformative scholars assume that
knowledge id not neutral but is influenced by human interests, that all
knowledge refelcts the power and social relationships within society, and that
an important purpose of knowledge construction is to help improve society:
(p.4). Back
to the top
Transformative
mixed methods design: (Creswell et al., Chapter 8, this
volume) This refers to a research project that Creswell, et al. describe as
follows: “In both perspective and outcomes, it is dedicated to promoting change
at levels ranging from the personal to the political. Furthermore, it is
possible to conduct any quantitative, qualitative, or mixed methods study with
a transformative or advocacy purpose.” Back to the top
Triangulation: The combinations and comparisons of multiple data sources, data collection and analysis procedures , research methods, or inferences that occur at the end of a study., Denzin (1978) used the terms data triangulation, theory triangulation and methodological triangulation. Erzberger and Udo have used the term to refer to agreement between inferences. See Rules of Integration. Back to the top
Two Phase Design:.
(Currall & Towler, Chapter 18, this volume) A study with a
qualitative phase followed by a quantitative phase or vice-versa. See Multistrand
Design. Back to the top
Validity (Measurement Validity, Data Validity): See Data Quality. Back to the top
Validity (Design Validity, Inferential Validity): See Inference Quality. Back to the top
Within-Design Consistency: The consistency of the procedures of the study from which the inferences emerged (Tashakkori & Teddlie, 2003, p. 717). See also Inference Quality. Back to the top