Dir. San Antonio Presentation

7/29/2019 Dir. San Antonio Presentation

1/47

MIXED RESEARCHMETHOD WITH

CONSENSUAL

QUALITATIVE RESEARCH(CQR)


2/47

Talk Outline

Clarifying Concepts

Mixing Research Methods

Quantitative

Qualitative

Purposes Steps

Data analysis process

Consensual Qualitative Research

Components

Data collection Data analysis

Auditing

Writing up

Advantages


3/47

Mixing Research ApproachesPractitioner Research and Evaluation Skills Training in Open and Distance Learning

(http://www.col.org/SiteCollectionDocuments/A5.pdf)

Multi-method studies use different methods

of data collection and analysis within a singleresearch paradigm.

Mixed method studies attempt to bring

together methods from differentparadigms.


4/47

Multi-method

designs

Mixed methods

designs

Use more than one

method but

restricted to

methods selectedfrom within one

worldview (i.e.

quantitative or

qualitative

approaches).

Use and mix both

qualitative and

quantitative data

or methods.


5/47

Comparison of quantitative and

qualitative research approaches

QUANTITATIVE QUALITATIVE

Purpose To study relationships, cause andeffect

To understand social phenomena

Design Developed prior to study Evolves during study

Approach Deductive; tests theory Inductive; generates theory

Tools Uses standardized measurement Uses face-to-face interaction

Sample Uses large samples Uses small samples

Analysis Statistical analysis of numericdata

Narrative description and

interpretation


6/47

Research Criteria(Lincoln & Guba, 1985)

Validity

Generalizability (thrusampling)

Reliability

(replicable)

Objectivity (limits

bias)

Credibility

(believable) Transferability (thru

thick description)

Dependability

(context of research) Reflexivity (examine

own biases)

Quantitative Qualitative


7/47

Why mix methods?

Mixed method research, apart fromavoiding unimethod bias, capitalizes onthe strengths of quantitative and

qualitative methods to provide a morecomplete understanding of theresearch problem and/or address

multiple questions (Johnson &Christensen, 2004; Wiersma & Jurs,2005).


8/47

Why mix methods?

Both quantitative andqualitative methods have

different but complementaryroles to play in a research

process and outcome(Sogunro, 2002, p. 3).


9/47

Purposes of Mixed Method

(Green, Caracelli and Graham,1989)

TriangulationComplementarityDevelopment InitiationExpansion


10/47

Triangulation

seeks convergence,

corroboration,correspondence of results

from the different methods.


11/47

Complementarity

seeks elaboration,

enhancement, illustration,and clarification of the

results from one method

with the results from the

other method.


12/47

Development

seeks to use the results from one

method to help develop or inform

the other method, wheredevelopment is broadly construed

to include sampling and

implementation, as well asmeasurement decisions.


13/47

Initiation

seeks the discovery of paradox

and contradiction, new

perspectives of frameworks, therecasting of questions or results

from one method with questionsor results from the other method.


14/47

Expansion

seeks to extend the breadth and

range of inquiry by usingdifferent methods for different

inquiry components (p. 259).


15/47

CITED

Complementarity

Initiation

Triangulation

Expansion

Development


16/47


17/47

Fundamental Principle of Mixed

Research (Turner, 2003)

Researchers should collect multiple

data using different strategies,

approaches, and methods in such a

way that the resulting mixture or

combination is likely to result in

complementary strengths and non-

overlapping weaknesses.


18/47

-s age m xe me o s a aanalysis process (Onwuegbuzie andTeddlie, 2003)

data reduction

data display

data transformationdata correlation

data consolidation

data comparison and

data integration


19/47

Data reduction

involves reducing the

dimensionality of the qualitative

data (e.g., via exploratory thematic

analysis, memoing) and

quantitative data (e.g., via

descriptive statistics, exploratory

factor analysis, cluster analysis).


20/47

Data display

Involves describing pictorially

the qualitative data (e.g.,

matrices, charts, graphs,networks, lists, rubrics, and

Venn diagrams) andquantitative data (e.g., tables,

graphs).


21/47

Data transformation

wherein quantitative data are

converted into narrative data that can

be analyzed qualitatively (i.e.,qualitized; Tashakkori & Teddlie,

1998) and/or qualitative data are

converted into numerical codes thatcan be represented statistically (i.e.,

quantitized; Tashakkori & Teddlie,

1998).


22/47

Data correlation

involves the quantitative data

being correlated with thequalitized data or the

qualitative data being

correlated with thequantitized data.


23/47

Data consolidation

Wherein both quantitative

and qualitative data arecombined to create new or

consolidated variables ordata sets.


24/47

Data comparison

involves comparing

data from the

qualitative and

quantitative data

source


25/47

Data integration

characterizes the final stage,

whereby both quantitative and

qualitative data are integratedinto either a coherent whole or

two separate sets (i.e.,qualitative and quantitative) of

coherent wholes.


26/47

Essential Components of CQR

open-ended questions in semi-

structured data collection techniques

(typically in interviews), several judges throughout the data

analysis process to foster multiple

perspectives; consensus to arrive at judgments

about the meaning of the data;


27/47

Essential Components of CQR

at least one auditor to check the work

of the primary team of judges and

minimize the effects of groupthink inthe primary team; and

domains, core ideas, and cross-

analyses in the data analysis.


28/47

Consensus

Is an integral part of the CQR method

(Hill et al., 1997)

relies on mutual respect, equalinvolvement, and shared power (p.

523).

a diversity of viewpoints is valued,honored, and protected (Williams &

Barber, 2004).


29/47

Expectations and Biases

Researchers should report both ..

expectations(beliefs that researchers have

formed based on reading the literature and

thinking about and developing researchquestions, Hill et al, 1997, p. 538) and

biases(personal issues that make it difficult

for researchers to respond objectively to thedata, p. 539)


30/47

Data Collection

Samples (8-15 participants)

Interviews (8-10 scripted questions

for 1 hour)Mode

taped telephone interviews,

taped face-to-face interviews,paper-and-pencil survey format, and

e-mail format


31/47

Data Analysis

Domains (i.e., topics used to group or clusterdata) are used to segment interview data.

Core ideas (i.e., summaries of the data that

capture the essence of what was said in fewerwords and with greater clarity) are used toabstract the interview data within domains.

Cross-analysis - to construct common themes

across participants (i.e., developing categoriesthat describe the common themes reflected inthe core ideas within domains across cases).

A diti


32/47

AuditingAuditors role is to

check.

whether the raw material is in the correct

domain,

that all important material has been

faithfully represented in the core ideas,

that the wording of the core ideas

succinctly captures the essence of the raw

data, and

that the cross-analysis elegantly and

faithfully represents the data.


33/47

Auditing.

The auditor thus provides

detailed feedback at each stage

of the analysis process (e.g.,

creating domains, constructing

core ideas, creating the cross-analysis).


34/47

Stability Check

Hill et al. (1997) recommend that after thedomains and core ideas are completed for allof the cases,

at least two cases be withheld from the initialcross analysis

and then used as a check to determinewhether all of the data for these cases fit into

the existing categories and whether the designations of general,

typical, and variant changed substantially withthe addition of the two new cases.


35/47

Charting Results

Charting results depicts visually the

relationships among categories across

domains, particularly for data representing

sequences of events (e.g., the process ofresolving a misunderstanding).

A criterion of at least three cases to

establish each connection betweendomains in the pathway.


36/47

Establishing the Trustworthiness

and Accuracy of the Data

use of participants to help assess

the accuracy and trustworthiness

of the data

sometimes called member

checking

W iti U th R lt


37/47

Writing Up the Results

and Discussion Sections

main purpose of the Results section is to

communicate the results clearly and

cogently to the audience.

results and conclusions of the data

analysis need to be logical,account for all

the data, answer the research questions

and make sense to the outside reader(Hill et al, 1997, p. 558).


38/47

Organizing Findings

according to their domains and

categories,

according to main groupings orclusters of the data, and

according to research questions.


39/47

Organizing Findings

Presenting core ideas,

Presenting participant quotes, and

Using a combination of core ideasand quotes to exemplify the

categories and subcategories either

in the text or in tables.


40/47

Organizing Findings

fully and richly describe at least

the general and typical categoriesand provide at least one example

(using the core ideas or quotes)

to illustrate each category in thetext.


41/47

Discussion

Use the Discussion section to

highlight the most important findings,

relate the results back to theliterature, and

pull the results together in some

meaningful way, perhaps bybeginning to develop theory to make

sense of the data.


42/47

Advantages of CQR

ideal for conducting in-depth studies of the inner

experiences of individuals.

good for studying events that are hidden from

public view, are infrequent, occur at varying timeperiods, have not been studied previously, or for

which no measures have been created.

CQR is ideal because it involves a rigorous

method that allows several researchers toexamine data and come to consensus about their

meaning, thus reducing the biases inherent with

just one person analyzing data


43/47

Limitations of CQR

the time commitment,

the repetitiousness of some of the tasks,

the lack of precise guidelines for some of

the steps (e.g., When have you collected

enough cases? How exactly do you come

to consensus?), and

the difficulty of combining findings across

studies

Features of Good Quality


44/47

Features of Good Quality

Research

Conceptual innovation

Methodological rigor

Rich, substantive content

3 questions guiding proposal


45/47

3 questions guiding proposal

reviewers

What are we going to learn as the

result of the proposed project that

we do not know now?Why is it worth knowing?

How will we know that the

conclusions are valid?


46/47

Ethics in Research

Research should not cause harm to the

participants

Researchers should obtain the informed

consent of participants

Researchers should respect the privacy of

participants

Researchers should avoid conflict of interest

Researchers should fairly and accurately

report their findings


47/47

Key References

Borrego, M., Douglas, E.P., & Amelink, C.T. (2009)Quantitative, Qualitative, and Mixed ResearchMethods in Engineering Education, Journal ofEngineering Education 98(1): 53-66.

Hill, C.E., Knox, S., Thompson, B.J., Williams,E.N., Hess, S.A., & Ladany, N. (2005) ConsensualQualitative Research: An Update, Journal ofCounseling Psychology52(2): 196-205.

Johnson, R. B. & Onwuegbuzie, A. J. (2004)Mixed Method Research: A Research ParadigmWhose Time Has Come, Educational Researcher33(7): 14-26.

Dir. San Antonio Presentation

Documents

Transcript of Dir. San Antonio Presentation