Thomas Salzberger, PD Dr. Quantitative Research Methods Closing session December 15, 2020 Starting at 15:15 ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Thomas Salzberger, PD Dr.
thomas.salzberger@wu.ac.at
Quantitative Research Methods
Closing session
December 15, 2020
Starting at 15:15
http://statmath.wu.ac.at/courses/m1bw/m1bw_en.html
Quantitative Studies
Research
STUDY DESIGN
problem,
Philosophy of science
Purpose of the
SAMPLING
study
MEASUREMENT DATA
Statistics
ANALYSIS
Psychometrics (Statistics; Metrology)
2 Conceptual theory (Concept of interest)
Successful measurement
“The road from scientific law to scientific measurement
can rarely be traveled in the reverse direction."
The Essential Tension: Selected Studies in Scientific Tradition
and Change (1977), p219. Thomas Kuhn
(1922-1996)
"Whether you can observe a thing or not depends on
the theory which you use. It is theory which decides
what can be observed.”
Albert Einstein to Werner Heisenberg during his
1926 Berlin lecture
Edward Fullbrook, “To observe or not to observe: Complementary pluralism in physics and
Ludwik Fleck
economics”, real-world economics review, issue no. 62, 15 December 2012, pp. 20-28, (1896 Lwiw - 1961)
http://www.paecon.net/PAEReview/issue62/Fullbrook62.pdf
A plea for good conceptual theories:
A qualitative theory of the construct to be measured is
to lead the way to successful measurement.
Normal science 'Normal science' means research firmly based upon one or more past scientific achievements, achievements that some particular scientific community acknowledges for a time as supplying Thomas Kuhn the foundation for its further practice. (1922-1996) The Structure of Scientific Revolutions (1962), p10. Paradigm for scientific research as contextualisation of a study One study is one piece in a series of studies
Hypothesis testing Conditional probabilities type-one error α: chance of rejecting the null hypothesis GIVEN it is true type-two error β: chance of retaining the null hypothesis GIVEN it is not true p-value: likelihood of the data (e.g. the difference observed or more extreme) GIVEN the null hypothesis Probability of the theory GIVEN the data/study requires a priori probability of the theory Bayes
Analysis In hypothesis testing, when the p-value is smaller (for example 0.03) than the designated significance level (for example alpha = 0.05), this implies ... □ [a] that the researcher retains the null hypothesis X [b] that the researcher accepts the alternative hypothesis □ [c] that the null hypothesis is wrong with a probability of 95% □ [d] that the alternative hypothesis is true with a probability of 97% A=theory, B=data P(A) a priori 1% -> P (A| B) a posteriori 14% a priori 10% -> a posteriori 64% a priori 50% -> a posteriori 94%
Covid Mass Testing
Sensitivity (correct positive) 96.7%
Specificity (correct negative) 99.9%
What is the chance of being infected, if I am tested positive?
What is the chance of being not infected, if I am tested negative?
After 2 days of testing Vienna: about 0.2% tested positive
Prevalence: 0.11%, in other words a priori not infected 99.89%
Truly neg Truly pos Sum
Tested neg 99.790 0.004 99.794
Tested pos 0.100 0.106 0.206
99.890 0.11 100
Tested neg: 99.996% neg (99.79/99.794), up from 99.89%
Tested pos: 51.6% pos (0.106/0.206), up from 0.11%Metaanalysis (in medicine)
Should pregnant women about to give birth too
early be given corticosteroids to contribute to
the development of the baby‘s lungs?
8Should pregnant women about to give birth
too early be given corticosteroids to
contribute to the development of the baby‘s lungs?
confidence interval line= no effect
* (significant!)
positive effect negative effect
* RCT: Randomized Controlled TrialShould pregnant women about to give birth
too early be given corticosteroids to contribute
to the development of the baby‘s lungs?
*
* RCT: Randomized Controlled TrialMeasurement
Over-reliance on a single self-report measure of a construct implies
which threat to construct validity?
□ [a] Mono-construct bias
X [b] Mono-method bias
□ [c] Mono-operation bias
□ [d] Mono-study bias
Mono-method bias refers to your measures or observations, not to your
programs or causes. (...) With only a single version of a self esteem
measure, you can’t provide much evidence that you’re really measuring
self esteem. Your critics will suggest that you aren’t measuring self esteem
– that you’re only measuring part of it, for instance. -> CONTENT
VALIDITY
Solution: try to implement multiple measures of key constructs and try to
demonstrate (perhaps through a pilot or side study) that the measures
you use behave as you theoretically expect them to. -> CONVERGENT
VALIDITY; EXTERNAL VALIDITY
11Measurement
Over-reliance on a single self-report measure of a construct implies
which threat to construct validity?
□ [a] Mono-construct bias
X [b] Mono-method bias
□ [c] Mono-operation bias
□ [d] Mono-study bias
Related issue: common-method bias
All constructs measured using the same method (e.g. self-administered
questionnaire).
Correlations are possible due to the same method.
-> Show that some correlations are close to zero. Use multiple/different
methods.
12Measurement
In the context of classical test theory (true score theory), which of
the following statements about reliability is/are correct?
Tick everything that applies!
X [a] Reliability is the ratio of true score variance and
observed score variance
X [b] Reliability is the degree to which a measure is
consistent
□ [c] Reliability is the degree to which the researcher can rely on the
observed scores reflecting what the researcher wants to measure
-> VALIDITY
□ [d] Reliability expresses the degree to which a measurement
instrument is easy to administer
13Scaling And Indexes
A Guttman scale …
Tick everything that applies!
X [a] is a cumulative model -> sum score meaningful
X [b] is a deterministic model -> no room for error
X [c] is a unidimensional model -> one dimension
□ [d] involves judges rating the items on a scale from 1 to 11
Note: Rasch model is a probabilistic version of the Guttman model
14Design
The regression toward the mean (regression threat) ...
Multiple options may be correct.
□ [a] is a consequence of an error in the data analysis
X [b] is a purely statistical phenomenon
□ [c] implies that each individual can only go up (or down) from the
pretest to the posttest -> only at the group level!
□ [d] can be ruled out when a true effect of the program
(intervention) exists
15with your
from
to
to
to
from
and finallyYou can also read
