[Bug]: Split-half coefficient unresponsive the standardized-unstandardized choice

[wolfvanpaemel]

JASP Version

0.19.3

Commit ID

No response

JASP Module

Reliability

What analysis are you seeing the problem on?

Split-half coefficient

What OS are you seeing the problem on?

Windows 10

Bug Description

in the classical unidimensional reliability option of the Reliability module, users can choose between the standardized and unstandardized version of their coefficients. however, it seems that the split-half coefficient is not responsive to this choice (and it seems to always reports the standardized version). this is probably not a software bug, but a psychometric oversight: just like there are two versions of alpha ("normal" alpha and standardized alpha, corresponding to the assumption of tau-equivalence and the assumption of parallellism, respectively), there are two versions of the split-half coefficient: guttman's lambda 4 and the spearman brown-coefficient, again corresponding to the assumption of tau-equivalence and the assumption of parallellism, respectively

see here from cho, 2016

Expected Behaviour

when a user wants the split half coefficient, report lambda 4 when unstandardized is selected, and report the spearman-brown coefficient when standardized is selected

Steps to Reproduce

just compute a split-half coefficient with any data set and check whether switching between standardized and unstandardized has any effect

Log (if any)

No response

More Debug Information

No response

Final Checklist

• I have included a screenshot showcasing the issue, if possible.
• I have included a JASP file (zipped) or data file that causes the crash/bug, if applicable.
• I have accurately described the bug, and steps to reproduce it.

[juliuspfadt]

@wolfvanpaemel thanks for creating the issue. There is actually a bug in the code. I fixed it now, so for the unstandardized case the coefficient is computed from the raw data, and for the standardized case the coefficient is estimated from the standardized data. In both cases we apply the Spearman Brown formula.

[wolfvanpaemel]

great. i will check it out in the next release

[juliuspfadt]

not sure if it will make it into the upcoming release (there was already a feature stop, but I hope so though :)

[wolfvanpaemel]

i think using the spearman brown equation in both cases is wrong.

think about the alpha coefficients. you can compute raw/unstandardized alpha based on 1) the raw data and 2) using the equation for raw/unstandardized alpha. you can compute standardized alpha based on 1) standardizing the data and 2) using the equation for raw alpha. JASP does this correctly.

the same holds for the split half coefficients: you can compute raw/unstandardized split half (i.e. lambda 4) based on 1) the raw data and 2) using the equation for lambda 4. you can compute standardized split half (i.e. spearman brown) based on 1) standardizing the data and 2) using the equation for lambda 4. JASP does NOT do that correctly. it should NOT use the Spearman Brown formula for Unstandardized Split half (just like it should not, and does not, use the Standardized Alpha formula for computing Unstandardized Alpha)

i created a new issue for this here

[EJWagenmakers]

Let me just emphasize the need to get this right (and add some unit tests so the correct solution is etched in stone). Also, it would be good to have some of this information in the help files, so that future users know what we implemented and why. Worth a blog post even, once it is 100% fixed