[Feature request]: Change split-half calculation regarding standardization

[wolfvanpaemel]

JASP Version

0.95.3

Commit ID

No response

JASP Module

Reliability

What analysis are you seeing the problem on?

split half coefficient

What OS are you seeing the problem on?

Windows 11

Bug Description

when I compute the Split Half coefficient using the Unstandardized option, JASP seems to report the Spearman Brown coefficient (at least, that is what my comparison with R and SPSS suggests; I did not look at the source code). The correct Unstandardized Split Half coefficient, however, is what SPSS calls Guttmans Split Half coefficient or alternatively Lambda 4. The Spearman Brown coefficient is the Standardized Split Half coefficient.

when I compute the Split Half coefficient using the Standardized option, I have no idea what JASP computes. It should be (but isn't) the Spearman Brown coefficient

sandboxL4.zip

Expected Behaviour

Unstandardized Split Half coefficient should yield Lambda 4. Standardized Split Half coefficient should yield Spearman Brown.

Steps to Reproduce

see jasp file

Log (if any)

No response

More Debug Information

No response

Final Checklist

• I have included a screenshot showcasing the issue, if possible.
• I have included a JASP file (zipped) or data file that causes the crash/bug, if applicable.
• I have accurately described the bug, and steps to reproduce it.

[juliuspfadt]

Thanks for creating the issue. I developed this together with Andries van der Ark, and we used the most basic formulas that we assumed reflect wha the coefficient name says, so might be we overlooked some conventions.

So we split the test into two parts, we build two vectors of sum scores for each part, we compute the correlation between both vectors, and we correct with the spearman-Brown prophecy. For the standardized coefficients we standardized the data beforehand and do the exact same thing

[juliuspfadt]

I asked Andries, it might be that we are using the basic solution from https://en.wikipedia.org/wiki/Spearman–Brown_prediction_formula and not the one for tau-equivalent reliability that follows Guttman.

[wolfvanpaemel]

indeed. the equation you need is 4S(X1,X2)/S2(X). that will lead to Lambda 4 aka Guttmans Split Half if you input the raw data and to Spearman Brown's Split Half if you input the standardized data.

[wolfvanpaemel]

the wikipedia article seems wrong too. the numerator of r_st should be S12 not R12 (see unidimensional split half tau-equivalent; this is from the same source as the wikipedia citres but apparently copied wrong)

update: i have corrected the wrong systematic expression for split-half tau-equivalent reliability on wikipedia

[wolfvanpaemel]

since unstandardized alpha reduces to guttman split half when k=2 and standardized alpha reduces to spearman-brown split half when k=2, it would make sense to have guttman split half when the unstandardized option is selected and spearman-brown split half when the standardized option is selected

but again, this behavior can be acheived with a single equation: 4s(x1,x2)/s2(x)

[juliuspfadt]

hmm so I agree that the help file would need to do a better job at highlighting what is done. However I am not certain as to these conventions about the split-half reliability. After asking Andries and thinking about this myself some more I wonder a few things:

• What exactly do you mean by "Spearman-Brown coefficient"?
• Lambda 4 is the maximum split-half reliability coefficient; that is, (1) compute the split-half reliability coefficient for all possible splits of the test (2) report the maximum value (e.g., Jackson & Agunwamba, 1977, https://doi.org/10.1007/BF02295979)
• The Split-half reliability coefficient in JASP is computed by (1) splitting the test in two halves, (2) computing the correlation between the sum scores of both halves, (3) increasing this correlation by using the Spearman-Brown prophecy formula for a test of double length (Lord and Novick, p. 84, Equation 4.2.10). That’s it. The help-file in JASP states how the test is split and what is correlated, it does not mention the prophecy formula (which it should).

So I am not sure that we are doing something wrong in JASP, it seems more to me that I addition to the very basic split-half correlation as it is in JASP there are other extensions. Now the question is rather if we want to implement them.

Of course, we could also opt for doing everything just like in SPSS.

I should note though that I included this exact split-half coefficient for Andries who wanted to have it for teaching psychometrics at the UvA.