There's an interesting exchange between a Mr McLean and the Mental Welfare Commission for Scotland (MWC) at What Do They Know.com?
Essentially, Mr McLean wishes to know "...the number of reported (alleged or proven) cases of abuse (physical, sexual, verbal, and mental) of detained/sectioned patients in Scotland reported by anybody." Okay, the request is strangely formal, and there is some ambiguity about how aspects of abuse are defined, but some might consider this a reasonable request. One of the MWC's statutory duties is to protect the interests of those with mental disorder in Scotland (although the Commission tend to focus on detained patients). Their website states that they believe that everyone with a mental illness should: "have the right to live free from abuse, neglect or discrimination". Nothing to disagree with that statement.
Mr McLean has made a request for numbers (not patient details, obviously) of cases of abuse of detained patients in Scotland. Since detained patients come very much within the remit of the Commission, one might imagine that they would/ should know how many cases there have been involving alleged abuse. Of course, if the Commission haven't been notified then they might not have such information, but they should be able to provide the number of cases reported to the Commission in recent years.
However, the Commission has invoked section 12(1) of the Freedom of Information (Scotland) Act 2002 which means that they consider it too costly to provide such information. There is a general limit of £600, so if a public body thinks it will cost more than £600 to find the information, they don't have to provide it.
Of course, there may be some people who think that a public body whose duties include monitoring of abuse of people with mental illness should be able to count the number of cases of abuse of people with mental illness on its records without it costing more than £600. The Commission will probably have some kind of database, and as most people know, a database that can't output such simple counts of relevant records isn't up to much. From what I've heard, the Commission has adopted a relatively 'paperless' office and forms and documents tend to be scanned into the system.
Despite some of the ambiguous language of the request, I don't think it should be too hard to provide Mr McLean with the numbers of cases of abuse (however defined) of detained patients in Scotland that it has on its records, and the argument that it would cost too much seems more than a little spurious.
Saturday, 1 January 2011
Tuesday, 21 December 2010
Terrible 'science' from the Scottish Government
Some recent evaluations of interventions or associations with close ties to the current mental health agenda in Scotland have highlighted the shift away from 'evidence-based policy' to 'policy-based evidence'.
The first was "An Evaluation of wellness planning in self-help and mutual support groups", produced by the Scottish Centre for Social Research, and published in September 2010. Essentially, this was an attempt to determine the benefits from WRAPs (Wellness Recovery Action Plans), which are advocated by the Scottish Government and are tightly integrated with the Recovery model which is driving most aspects of mental health service delivery in the UK and beyond.
The remit for the evaluation was: "...to assess the relevance, impact and effectiveness of Wellness Recovery Action Planning (WRAP) as a tool for self management and wellness planning by individuals with mental health problems from pre-existing and newly-formed groups, where the possibilities for continued mutual support in the development of WRAPs could be explored."
The approach was 'mixed methods', which typically combines qualitative and quantitative methods to address different aspects of the particular research question. Whilst some quantitative information was collected, the bulk of the evaluation relied on qualitative information from interviews and focus groups.
As with most such evaluations of policy, there is a dissociation between the objectives and methods. The purpose of the evaluation included a specific aim to determine the "effectiveness" of WRAPS, but data on whether they make a difference is notably absent. There are lots of anecdotes on how great everything is, but it's when the researchers try to get to grips with some numbers that the problems really start.
Table 2 is shown below. It compares scores on the Recovery Assessment Scale (RAS) and the Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS) in people before and after their WRAP training:
The authors report that: "...that RAS scores increased in all groups, and WEMWBS scores in all but one group, after the respondents had completed their WRAP training. This suggests that both the facilitators and participants had more positive views in relation to their own sense of recovery and well-being having been trained..."
Also: "...the pre- and post- WRAP training questionnaires were not completed by the same number of people. Any differences between pre- and post- WRAP training scores might, therefore, be due to the fact that people with higher scores completed the post-WRAP training questionnaires."
Indeed. If the authors looked at the table, they might have noticed that when the pre- and post-groups were the same size, the change is not really that great. The numbers in each group are very small, but any changes are likely to be non-significant (i.e. due to chance alone). The only groups that show improvements are those where some people did not return forms. Some groups (e.g. the Tayside Carers Groups 1 and 2) had more respondents in the post-group than existed in the pre-group. This suggests poor handling of data and/ or the possibility that different people were completing forms in the pre- and post groups.
In the groups where data is missing in the post-group, the lower end of the range will often shift up whilst the upper end doesn't move. This suggests that those who had low scores on pre-testing are not included in post-testing. One can suggest this because a change on the WEMWBS score of 10 is improbable given the psychometric properties of the scale, although very little is actually known about its sensitivity to change.
The authors should have known that averaging very small groups is problematic: the average of two people with scores of 2 and 100 on a 100-point scale is 51. However, this score is representative of neither of them. Trying to calculate the mean for such small samples is simply daft, and betrays a degree of statistical ignorance. This doesn't stop the researchers from spinning the poor data: "However, these results do support the very positive views expressed by facilitators and group participants in the main qualitative phase of the study."
The really low quality of the data analysis is indicated by the scores of the facilitators pre- and post-WRAP training. These are shown below (as Table 1):
The authors helpfully point out that scores on the WEMWBS can range from 14-70. It is therefore puzzling how the Pre-WRAP WEMWBS scores range from 76-100 (all above the upper end of the scale), yet still have an average score of 49.4. This is, of course, impossible given the range of scores.
Unfortunately, this is not a new phenomena in Government-commissioned evaluations of its own policy. The quantitative evaluation is handled as if the researchers didn't have a GCSE in maths, and yet all the conclusions are positive. A further example of this will follow.
More papers on the WEMWBS can be found on the links below.
TENNANT, R., FISHWICK, R., PLATT, S., JOSEPH, S. & STEWART-BROWN, S. (2006) Monitoring positive mental health in Scotland: validating the Affectometer 2 scale and developing the Warwick-Edinburgh Mental Well-Being Scale for the UK. Edinburgh, NHS Health Scotland, University of Warwick and University of Edinburgh.
TENNANT, R., HILLER, L., FISHWICK, R., PLATT, S., JOSEPH, S., WEICH, S., PARKINSON, J., SECKER, J. & STEWART-BROWN, S. (2007) The Warwick-Edinburgh Mental Well-being Scale (WEMWBS): development and UK validation. Health and Quality of Life Outcomes, 5, 63.
STEWART-BROWN, S., TENNANT, A., TENNANT, R., PLATT, S., PARKINSON, J. & WEICH, S. (2009) Internal Construct Validity of the Warwick-Edinburgh Mental Well-being Scale (WEMWBS): a Rasch analysis using data from the Scottish Health Education Population Survey. Health and Quality of Life Outcomes, 7, 15.
The first was "An Evaluation of wellness planning in self-help and mutual support groups", produced by the Scottish Centre for Social Research, and published in September 2010. Essentially, this was an attempt to determine the benefits from WRAPs (Wellness Recovery Action Plans), which are advocated by the Scottish Government and are tightly integrated with the Recovery model which is driving most aspects of mental health service delivery in the UK and beyond.
The remit for the evaluation was: "...to assess the relevance, impact and effectiveness of Wellness Recovery Action Planning (WRAP) as a tool for self management and wellness planning by individuals with mental health problems from pre-existing and newly-formed groups, where the possibilities for continued mutual support in the development of WRAPs could be explored."
The approach was 'mixed methods', which typically combines qualitative and quantitative methods to address different aspects of the particular research question. Whilst some quantitative information was collected, the bulk of the evaluation relied on qualitative information from interviews and focus groups.
As with most such evaluations of policy, there is a dissociation between the objectives and methods. The purpose of the evaluation included a specific aim to determine the "effectiveness" of WRAPS, but data on whether they make a difference is notably absent. There are lots of anecdotes on how great everything is, but it's when the researchers try to get to grips with some numbers that the problems really start.
Table 2 is shown below. It compares scores on the Recovery Assessment Scale (RAS) and the Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS) in people before and after their WRAP training:
The authors report that: "...that RAS scores increased in all groups, and WEMWBS scores in all but one group, after the respondents had completed their WRAP training. This suggests that both the facilitators and participants had more positive views in relation to their own sense of recovery and well-being having been trained..."
Also: "...the pre- and post- WRAP training questionnaires were not completed by the same number of people. Any differences between pre- and post- WRAP training scores might, therefore, be due to the fact that people with higher scores completed the post-WRAP training questionnaires."
Indeed. If the authors looked at the table, they might have noticed that when the pre- and post-groups were the same size, the change is not really that great. The numbers in each group are very small, but any changes are likely to be non-significant (i.e. due to chance alone). The only groups that show improvements are those where some people did not return forms. Some groups (e.g. the Tayside Carers Groups 1 and 2) had more respondents in the post-group than existed in the pre-group. This suggests poor handling of data and/ or the possibility that different people were completing forms in the pre- and post groups.
In the groups where data is missing in the post-group, the lower end of the range will often shift up whilst the upper end doesn't move. This suggests that those who had low scores on pre-testing are not included in post-testing. One can suggest this because a change on the WEMWBS score of 10 is improbable given the psychometric properties of the scale, although very little is actually known about its sensitivity to change.
The authors should have known that averaging very small groups is problematic: the average of two people with scores of 2 and 100 on a 100-point scale is 51. However, this score is representative of neither of them. Trying to calculate the mean for such small samples is simply daft, and betrays a degree of statistical ignorance. This doesn't stop the researchers from spinning the poor data: "However, these results do support the very positive views expressed by facilitators and group participants in the main qualitative phase of the study."
The really low quality of the data analysis is indicated by the scores of the facilitators pre- and post-WRAP training. These are shown below (as Table 1):
The authors helpfully point out that scores on the WEMWBS can range from 14-70. It is therefore puzzling how the Pre-WRAP WEMWBS scores range from 76-100 (all above the upper end of the scale), yet still have an average score of 49.4. This is, of course, impossible given the range of scores.
Unfortunately, this is not a new phenomena in Government-commissioned evaluations of its own policy. The quantitative evaluation is handled as if the researchers didn't have a GCSE in maths, and yet all the conclusions are positive. A further example of this will follow.
More papers on the WEMWBS can be found on the links below.
TENNANT, R., FISHWICK, R., PLATT, S., JOSEPH, S. & STEWART-BROWN, S. (2006) Monitoring positive mental health in Scotland: validating the Affectometer 2 scale and developing the Warwick-Edinburgh Mental Well-Being Scale for the UK. Edinburgh, NHS Health Scotland, University of Warwick and University of Edinburgh.
TENNANT, R., HILLER, L., FISHWICK, R., PLATT, S., JOSEPH, S., WEICH, S., PARKINSON, J., SECKER, J. & STEWART-BROWN, S. (2007) The Warwick-Edinburgh Mental Well-being Scale (WEMWBS): development and UK validation. Health and Quality of Life Outcomes, 5, 63.
STEWART-BROWN, S., TENNANT, A., TENNANT, R., PLATT, S., PARKINSON, J. & WEICH, S. (2009) Internal Construct Validity of the Warwick-Edinburgh Mental Well-being Scale (WEMWBS): a Rasch analysis using data from the Scottish Health Education Population Survey. Health and Quality of Life Outcomes, 7, 15.
Wednesday, 3 November 2010
Psychiatry Ethics Film Festival - Do the films really do justice to their aims?
There's a film festival in Edinburgh running from 26 November to the 28 November. It's called the "Psychiatry Ethics Film Festival".
It raises some questions: "Should individuals with certain mental health problems be placed, against their will, in psychiatric institutions? What are the causes of some of these mental disorders? Do some of the disorders run in families? In other words, is it genetic? Is surgery on the brain of a patient possible? Should it take place again their will?"
All interesting questions, and it will "seek to answer some of these questions while leading post-film debates with an ethics expert. "
So, what are the films? (links go to the film festival website [FFW], or the Internet Movie Database [IMDB]):
Well, The Eighth Day is about someone with Down's Syndrome (Georges) and their friendship with a man who is down on his luck (Harry). No problem with that. It's just that we already know that Down's syndrome is genetic. Although Georges is on the run from an institution, it's not really clear that the story is about this. The story is about the development of the friendship between the two characters.
Girl, Interrupted is set in an institution...in America...in the 1960s. Of questionable relevance to British psychiatric hospitals in the 21st Century, perhaps. The main character (played by Winona Ryder) gets a diagnosis of "borderline personality", and the other key protagonist (played by Angelina Jolie) demonstrates a range of rather dissocial personality traits. It's based on a book by Susanna Kaysen and her experiences in a psychiatric hospital in the late 1960s. At the time, she was given diagnoses of "psychoneurotic depressive reaction" and "personality pattern disturbance, mixed type".
It's difficult to know how relevant or informative this film really is. In a review of the book and film in the journal Psychiatric Services, Dr Jeffrey Geller suggests, "historical accuracy about borderline personality disorder is abandoned. At the time Ms. Kaysen was in the hospital, one could not simply take a book of diagnostic criteria off the shelf, as occurs in the film, and read about "borderline personality". The film needs this distortion to make itself whole." It's undoubtedly a Hollywood creation, and perhaps tells us more about the Hollywood portrayal of mental illness than mental illness itself.
The Madness of King George is a strange choice. Admittedly, King George III displays some unusual behaviour. The problem is that there is a possibility that he suffered from one of a range of physical illnesses. One theory is that he had 'variegate porphyria' [Paper Download], a genetic illness which did seem to exist in his family. Another is that he suffered from arsenic poisoning [Paper Download]. Both are rare conditions that are likely to tell us much about the ethical issues affecting psychiatric services today.
Onto Stephen Fry: The Secret Life of the Manic Depressive. Stephen Fry has become the celebrity voice of Bipolar Disorder. The festival website actually calls it 'Manic Depression', an outdated term. However, I'm not entirely convinced that his experience is representative of many people with Bipolar Disorder. His high level of functioning, whilst not a clear exclusion criteria, is rare for many people with bipolar disorder in community mental health teams. Additionally, he appears to be on no medication yet he is told that he has a severe case. To maintain such prolonged media activity and level of functioning with a severe bipolar disorder, yet be on no medication is difficult to reconcile for most psychiatrists who see patients with affective disorders. Again, one wonders how his case can be matched to some of the aims of the film festival.
The only film which would seem to be of real people with real mental illness in contemporary times is Sectioned, which was shown on terrestrial TV some time ago. The other documentary (Mental: A History of the Madhouse) is about the closure of mental asylums in the UK following the Second World War. Yes, it does address mental institutions but it remains to be seen how relevant the experience of people 50 years ago is when trying to address such ethical issues in a completely different Health Service and with an entirely different system of Mental Health legislation.
I suppose that those behind the festival should be credited with not showing One Flew over the Cuckoo's Nest, but this is probably one of the only films which shows psychiatric neurosurgery (albeit lobotomy which hasn't been used for decades). The other film showing psychosurgery was Frances, starring Jessica Lange. This was apparently based on the life of Frances Farmer and was based on the 1978 book 'Shadowland' by William Arnold. All well and good, except that it turns out he made a lot of it up, and Frances Farmer never had psychosurgery.
So, the festival has some laudable aims, but the films all have substantial problems and few actually relate to the questions that the festival is attempting to answer. By raising important questions, but showing what is arguably a poor choice of films, I'm not convinced that the festival is doing much to expand our knowledge...
It raises some questions: "Should individuals with certain mental health problems be placed, against their will, in psychiatric institutions? What are the causes of some of these mental disorders? Do some of the disorders run in families? In other words, is it genetic? Is surgery on the brain of a patient possible? Should it take place again their will?"
All interesting questions, and it will "seek to answer some of these questions while leading post-film debates with an ethics expert. "
So, what are the films? (links go to the film festival website [FFW], or the Internet Movie Database [IMDB]):
- The Eighth Day [IMDB]
- Girl, Interrupted [IMDB]
- Mental + Sectioned Double Bill [FFW]
- Stephen Fry: The Secret Life of the Manic Depressive [FFW]
- The Madness of King George [IMDB]
Well, The Eighth Day is about someone with Down's Syndrome (Georges) and their friendship with a man who is down on his luck (Harry). No problem with that. It's just that we already know that Down's syndrome is genetic. Although Georges is on the run from an institution, it's not really clear that the story is about this. The story is about the development of the friendship between the two characters.
Girl, Interrupted is set in an institution...in America...in the 1960s. Of questionable relevance to British psychiatric hospitals in the 21st Century, perhaps. The main character (played by Winona Ryder) gets a diagnosis of "borderline personality", and the other key protagonist (played by Angelina Jolie) demonstrates a range of rather dissocial personality traits. It's based on a book by Susanna Kaysen and her experiences in a psychiatric hospital in the late 1960s. At the time, she was given diagnoses of "psychoneurotic depressive reaction" and "personality pattern disturbance, mixed type".
It's difficult to know how relevant or informative this film really is. In a review of the book and film in the journal Psychiatric Services, Dr Jeffrey Geller suggests, "historical accuracy about borderline personality disorder is abandoned. At the time Ms. Kaysen was in the hospital, one could not simply take a book of diagnostic criteria off the shelf, as occurs in the film, and read about "borderline personality". The film needs this distortion to make itself whole." It's undoubtedly a Hollywood creation, and perhaps tells us more about the Hollywood portrayal of mental illness than mental illness itself.
The Madness of King George is a strange choice. Admittedly, King George III displays some unusual behaviour. The problem is that there is a possibility that he suffered from one of a range of physical illnesses. One theory is that he had 'variegate porphyria' [Paper Download], a genetic illness which did seem to exist in his family. Another is that he suffered from arsenic poisoning [Paper Download]. Both are rare conditions that are likely to tell us much about the ethical issues affecting psychiatric services today.
Onto Stephen Fry: The Secret Life of the Manic Depressive. Stephen Fry has become the celebrity voice of Bipolar Disorder. The festival website actually calls it 'Manic Depression', an outdated term. However, I'm not entirely convinced that his experience is representative of many people with Bipolar Disorder. His high level of functioning, whilst not a clear exclusion criteria, is rare for many people with bipolar disorder in community mental health teams. Additionally, he appears to be on no medication yet he is told that he has a severe case. To maintain such prolonged media activity and level of functioning with a severe bipolar disorder, yet be on no medication is difficult to reconcile for most psychiatrists who see patients with affective disorders. Again, one wonders how his case can be matched to some of the aims of the film festival.
The only film which would seem to be of real people with real mental illness in contemporary times is Sectioned, which was shown on terrestrial TV some time ago. The other documentary (Mental: A History of the Madhouse) is about the closure of mental asylums in the UK following the Second World War. Yes, it does address mental institutions but it remains to be seen how relevant the experience of people 50 years ago is when trying to address such ethical issues in a completely different Health Service and with an entirely different system of Mental Health legislation.
I suppose that those behind the festival should be credited with not showing One Flew over the Cuckoo's Nest, but this is probably one of the only films which shows psychiatric neurosurgery (albeit lobotomy which hasn't been used for decades). The other film showing psychosurgery was Frances, starring Jessica Lange. This was apparently based on the life of Frances Farmer and was based on the 1978 book 'Shadowland' by William Arnold. All well and good, except that it turns out he made a lot of it up, and Frances Farmer never had psychosurgery.
So, the festival has some laudable aims, but the films all have substantial problems and few actually relate to the questions that the festival is attempting to answer. By raising important questions, but showing what is arguably a poor choice of films, I'm not convinced that the festival is doing much to expand our knowledge...
Subscribe to:
Posts (Atom)