[R-sig-ME] Analysis of signal detection data
Ken Knoblauch
ken.knoblauch at inserm.fr
Mon Nov 15 09:03:13 CET 2010
David Duffy <davidD at ...> writes:
> On Sun, 14 Nov 2010, Thompson,Paul wrote:
> >> This situation is well described and most appropriately analyzed by the
> >> use of GEE methods.
> > Mike Lawrence wrote:
> > Yet another query on whether traditional stats employed in psychology
> > might be improved by mixed effects modelling...
> > [diagnostic accuracy v. gold standard]
> As I understand it, GEE can be seen as an approximate method related to
> approaches such as quasi-likelihood, though there may be cases where its
> underlying model is closer to the true state of what you are actually
> looking at. So, a GLMM may better -- I'm more familiar with SEM/
> graphical models for this type of setup, in that they are more flexible
> eg diagnosis where there is no gold standard diagnosis to compare to,
> mixture models for task difficulty...
> These will usually have a "full likelihood" GLMM underlying them, or some
> type of approximation eg Browne's WLS seen in Lisrel/MX.
>
> Just 2c, David Duffy.
The classic signal detection theory (SDT) model,
in which an observer's choice behavior is modeled by an
unobserved decision variable contaminated by noise, is
essentially a probit model (see Green & Swets and
Macmillan & Creelman references below). A logistic
version called choice theory was developed by Luce.
A clear formalization in terms of GLMs is given in the
DeCarlo reference below (but see also the Klein reference
for a general extension to psychometric function fitting
and the 2 papers by Wichmann & Hill in the same special
issue) with extensions to rating scales using ordinal
regression.
In R, these models can be simply fit with a glm, taking
the observer's response as a 2-level variable (factor or
logical) and the stimulus (signal present/absent) as a
2-level factor. The model would be
g(E(Pr(Resp = 1))) = beta_0 + beta_1 Stim,
where g is the link function, taken as an inverse
Gaussian for the probit case. When, the stimulus is
absent, the linear predictor gives beta_0 which
equals Phi^-1(Pr(False Alarm)) = z(FA) (to use the
more conventional SDT terminology). Treatment contrasts
then directly give you z(Hit) - z(FA) = d'.
For example,
Stim <- rep(factor(c(0, 1), labels = c("A", "P")),
each = 500)
##unbiased observer w/ true d' = 1.0
Resp <- ((Stim == "P") + rnorm(length(Stim))) > 0
glm(Resp ~ Stim, binomial(probit))
Call: glm(formula = Resp ~ Stim, family = binomial(probit))
Coefficients:
(Intercept) StimP
-0.05015 1.03642
Degrees of Freedom: 999 Total (i.e. Null); 998 Residual
Null Deviance: 1283
Residual Deviance: 1135 AIC: 1139
Compare this with the manual method of calculating d'
given in SDT texts.
( dp.tab <- prop.table(table(Stim, Resp), margin = 1) )
Resp
Stim FALSE TRUE
A 0.520 0.480
P 0.162 0.838
diff(qnorm(dp.tab[, 2]))
P
1.036425
The extension to more complex GLM's with more complex
explanatory variables as well as to miwed-effects models
with glmer then follows quite naturally.
(You can't extend this to m-alternative forced-choice
in a simple way, i.e., you cannot simply substitute
one of the links from the psyphy package in the glm.
The situation is more complicated than that.)
With glmer you can add observer and item random effects.
If you had several observers in the above toy example,
you could include terms (1 | Obs) for random criterion
models and (Stim | Obs) to model both random criterion
and random sensitivity (using SDT jargon, here).
The sdtalt package provides a simplified interface
(wrapper function) for specifying response, stimulus,
observer and item variables although the last time that I
looked at the code, the Resp and Stim (or equivalent)
variables were reversed in the call to glmer.
See also, the MLDS and MLCM packages for extensions of
an SDT model to choices involving suprathreshold stimuli
@Book{GreenSwets66,
author = {D. M. Green and J. A. Swets},
title = {Signal Detection Theory and Psychophysics},
publisher = {Robert E. Krieger Publishing Company},
year = {1966},
address = {Huntington},
}
@Book{MacCreel,
author = {N.~A.~Macmillan and C.~D.~Creelman},
title = {Detection Theory: A User's Guide},
publisher = {Lawrence Erlbaum Associates},
year = {2005},
address = {New York},
edition = {Second},
}
@Article{DeCarlo98,
author = {L. T. De{C}arlo},
title = {Signal detection theory and generalized linear models},
journal = {Psych Meth},
year = {1998},
volume = {3},
pages = {186--205},
}
@Article{Klein01,
Author="Klein, S. A. ",
Title="{{M}easuring, estimating, and understanding
the psychometric function: a commentary}",
Journal="Perception \& Psychophysics",
Year="2001",
Volume="63",
Pages="1421--1455",
Month="Nov"
}
More information about the R-sig-mixed-models
mailing list