Reliability of diagnosis v validity of diagnosis

 Reliability of diagnosis v validity of diagnosis
Posted: Tuesday, March 13, 2018 @ 08:28 PM
Joined: 20 July 2009
Posts: 118

1. If the DSM and ICD lead to a different diagnosis can this be an issue of reliability? I have always assumed not. I interpret this as a validity issue (concurrent validity). Some textbooks are getting me confused though. I feel that reliability of diagnosis refers to whether the same set of symptoms would get the same diagnosis from different psychiatrists. (inter rater-reliability) A diagnosis is also reliable if the diagnosis is consistent over two different time periods for the same patient using the same psychiatrist. (test-retest reliability). So reliability is about consistency of diagnosis between psychiatrists not more than one classification system?

2. Rosenhan - I have seen inconsistences in published sources, websites and mark schemes as to whether Rosenhan's study shows problems with the reliability of diagnosis. My understanding and interpretation is the study shows good reliability using DSM (II?) as 7/8 of the diagnoses were the same (Schizophrenia in remission)  however shows low validity as the diagnosis was incorrect (although Spitzer does counter argue this).

Would really like your views on this.

Posted: Tuesday, March 13, 2018 @ 11:40 PM
Joined: 24 March 2011
Posts: 100

Hi Tim

If the lines of discussion go back far enough, there was a really useful discussion on reliability and validity started by Stacey on 17th May 2015 with the last posting being by Caroline Ball on 30th May 2015.

I would agree that reliability is demonstrated by the same symptoms from one person being independently given the same diagnosis by two different psychiatrists using the same diagnostic criteria (although that doesn't mean the diagnosis is valid).

I wouldn't like to put my head on a block re DSM and ICD in terms of whether agreement between the two constitutes concurrent validity, because concurrent validity refers to the outcome of comparing an established measure with a newer one - DSM and ICD are different from one another but one is not intended to replace the other (at least, I don't think this is the case).

Regarding over time - test-test reliability (assuming same procedure and same participants) is only relevant if you are talking about a characteristic that can reasonably be considered to be stable e.g. adult IQ. However, "over time" is probably open to some debate as to when the time span is so long that it becomes validity over time rather than test-retest reliability.

As far as Rosenhan is concerned, I have also seen a great deal of inconsistency in the way different specifications interpret it, and it may be a good idea to ask your exam board what they think about it. However, if you read the original research paper, it is clear that Rosenhan's purpose in conducting the study was to demonstrate a validity issue - that there was no objective truth in the definition of abnormality and, once labelled, then all behaviour was interpreted according to the label.

Posted: Wednesday, March 14, 2018 @ 05:17 PM
Joined: 1 November 2007
Posts: 2061

Rosenhan is reliable 7/8 pseudo-patients presenting with the same symptoms had same diagnosis.
Not valid as wrongly diagnosed

There are some studies that have looked at reliability of DSM and ICD by sending same descriptions to practitioners using either ICD or DSM (Nicholls I think) so you can check reliability between the two manuals. From memory there was 68% agreement for most disorders.

Posted: Saturday, March 17, 2018 @ 10:06 PM
Joined: 16 January 2009
Posts: 143

Hi Tim everything you have said is in-keeping with what I teach for Edexcel and have been teaching for donkeys years. Great to see the comments from Marmaduke though, thank you for those! As you know, I am often concerned that what we teach simply 'scratches the examination itch' rather than what is necessarily correct ;).