Alcohol and cannabis use are relatively common among adolescents in developed countries, and they have detrimental effects on health, relationships, and educational and occupational achievement. Universal prevention interventions have the potential to reduce substance use if delivered to adolescents before they become involved with these substances.
One such intervention, ‘Climate Schools’ (explained below) leads to improvements in knowledge about alcohol and cannabis, and reductions in use of those substances, six and 12 months after delivery of the intervention (Newton et al., 2009, 2010).
The aim of the present study was to investigate whether the intervention would also have a beneficial effect on variables that are risk-factors for alcohol and cannabis use. Although there are many risk-factors, the authors of this paper focused on truancy (skipping school), psychological distress, and moral disengagement (‘the tendency to disengage from moral self-control and responsibility that ordinarily governs behaviour’). They predicted that each of these variables would be positively affected by the Climate Schools intervention.
Universal prevention interventions have the potential to reduce substance use, if delivered to adolescents before they start taking drugs. Image: Carol VanHook CC BY-NC-SA 2.0
Method
This is a cluster randomized controlled trial (RCT) that recruited 764 students (mean age 13.1) from 10 secondary schools in Sydney, Australia in 2007 and 2008. Schools were randomly assigned to receive either their usual health classes (including drug and alcohol education) or the Climate Schools: Alcohol and Cannabis Course intervention.
The intervention included a total of 12 x 40 minute sessions, with six sessions delivered initially and the following six sessions delivered six months later. Each session included a 15-20 minute internet component involving a cartoon storyline about experiences of alcohol and cannabis use that students completed individually. The second part of each session was delivered by teachers, who aimed to reinforce the information provided in the cartoons.
All participants completed self-report measures of alcohol and cannabis use, and truancy, psychological distress and moral disengagement at baseline, immediately after the intervention, and six- and 12-months after the end of the intervention.
The report implies that this is a novel study which aims to extend the results from the previously-reported trial, in which beneficial effects of the Climate Schools intervention on alcohol and cannabis use were reported (Newman et al., 2009, 2010). However this isn’t a unique study at all, but rather an analysis of secondary outcomes that were recorded as part of the trial but were not reported in the original articles.
Such ‘salami slicing’ of trials is not ideal but is an inevitable consequence of tight word limits imposed by academic journals, which make it difficult to report all of the results from complex trials in a single article. Nonetheless, this Elf was a little irritated that the authors were not upfront about the fact that this report describes analysis of secondary outcomes from a trial that has already been reported. Such practices give the impression that the trial has been more intensively researched than it actually has.
Each session included a internet component involving a cartoon storyline about experiences of alcohol and cannabis use that students completed individually. The second part was delivered by teachers, who aimed to reinforce the information provided in the cartoons.
Each session combined an Internet-based component, which was followed up with the key messages being reinforced by a teacher.
Image: Enokson CC BY-NC-SA 2.0
Results
Truancy
Climate and Control groups did not differ immediately after the intervention (p = .14) or at six-month follow-up (p = .30). However, truancy rates were significantly lower in the Climate compared to the Control group at 12-month follow-up (F(1, 374) = 5.47, p < .02). Although within-group contrasts were not reported, inspection of the mean values suggests that truancy rates remained constant over time in the Climate group, but tended to increase over time in the Control group.
Psychological distress
Climate and Control groups did not differ immediately after the intervention, although the trend was in the right direction (F(1, 336) = 3.11, p = .08). However, psychological distress was significantly lower in the Climate compared to the Control group at six-month follow-up (F(1, 398) = 4.48, p < .05), and this group difference was maintained at 12-month follow-up (F(1, 375) = 5.55, p < .02). Again, within-group contrasts were not reported but inspection of the mean values suggests a different pattern to that for truancy: psychological distress appeared to decline over the course of follow-up for the Climate group, but there was no reliable change in the Control group.
Moral disengagement
The between-group difference in this variable was significant at 12-month follow-up (F(1, 314) = 5.59, p < .02), with lower values in the Climate compared to the Control group. Groups did not differ immediately after the intervention or at six-month follow-up (p = .91 and p = .23, respectively). Inspection of the mean values suggests that moral disengagement tended to increase over time in the Control group, whereas in the Climate group it increased between post-intervention and six-month follow-up, but then decreased at 12-month follow-up.
Discussion
This paper is important because it ‘puts flesh on the bones’ of the previous findings from this trial (Newman et al., 2009, 2010) which showed that the Climate Schools intervention led to improvements in alcohol and cannabis knowledge that were accompanied by reductions in alcohol and cannabis use at six- and 12-month follow-ups.
These new analyses demonstrate that the previously-reported behavioural changes were accompanied by reductions in truancy, psychological distress, and moral disengagement. As implied by the authors, it is plausible that reductions in these variables, which are all risk-factors for substance use, may have mediated the behavior change that occurred as a result of the intervention.
Limitations
The authors noted a number of limitations of their study including the use of online self-report measures, representativeness of the schools and baseline differences (including in truancy and psychological distress) between groups. While these limitations are important (particularly the baseline differences), this Elf feels that other limitations were not flagged up at all.
Firstly, the authors reported between-group contrasts in outcome measures but not within-group contrasts, and inspection of the means suggests some potentially interesting within-group changes in the outcome measures that were not comparable for all variables.
Secondly, given that the Climate intervention can reduce risk-factors for substance use and actual substance use behaviour, then it is plausible that the former change mediates the latter. The authors could have conducted such a mediation analysis and this seems like a missed opportunity.
Finally, there was no mention of these secondary hypotheses or these secondary outcome measures either in the published study protocol, or either of the two previous reports of findings from this trial (Newman et al., 2009, 2010). I was left wondering how many other measures the authors had administered, and how many other secondary hypotheses they had. If it turns out that they had collected a lot of additional secondary measures (that have not yet been reported), then this raises concerns regarding the use of multiple statistical tests without correction for multiple comparisons.
Conclusions
Climate schools, an internet-based universal prevention intervention for alcohol and cannabis use in schoolchildren, leads to reductions in alcohol and cannabis use that are accompanied by reductions in truancy, psychological distress and moral disengagement.
This is interesting and potentially promising research, but the ‘salami slicing’ that has taken place makes it hard to be certain about what we can take-away
Links
Newton, N. C., Andrews, G., Champion, K. E., & Teesson, M. (2014). Universal internet-based prevention for alcohol and cannabis use reduces truancy, psychological distress and moral disengagement: A cluster randomized controlled trial. Preventive Medicine 65, 109-115.
Newton, N. C., Andrews, G., Teesson, M., & Vogl, L. E. (2009). Delivering prevention for alcohol and cannabis using the internet: A cluster randomized controlled trial. Preventive Medicine, 48, 579-584.
Newton, N. C., Teesson, M., Vogl, L. E., & Andrews, G. (2010). Internet-based prevention for alcohol and cannabis use: final results of the Climate Schools Course. Addiction 105, 749-759.