Growl: Flawed Narconon Study Gets Published

Discussion in 'Narconon' started by XenuLovesU, May 3, 2008.

  1. Re: Growl: Flawed Narconon Study Gets Published

    OMG, that study's awful. What's also interesting is that many - myself included - aren't willing to put their names behind a critique of it's flaws, because of concerns about retribution from the study's affiliates.

    How would Anonymous feel about submitting an anonymous response to - and detailed - critique of the study to the journal? I know that academic papers are not usually anonymous: however, we could challenge this on the grounds that if a journal publishes a terrible article where people are (predictably) reluctant to criticise the methodology due to concerns about retribution from affiliates, the 'normal' rules of academic debate can no longer apply. Davey's Cat's response to the article is excellent - but I like the idea of having something much longer, going through the article in painstaking detail, to point out how error-ridden it is.

    I'm happy to contribute my limited skills, and I know others who might be interested in working on this... I was amazed that (as Davey's Cat suggests) Table 5 actually seems to use 'more' and 'less' in a way that's inconsistent with the figures. And seems to be measuring 'success' in terms of getting young people to believe untrue statements about drugs - for example, while alcohol obviously carries risks, there is not good evidence that 1-2 drinks per day is a high risk lifestyle choice...
  2. Davey's Cat Member

    Re: Growl: Flawed Narconon Study Gets Published

    Excellent idea - I'm happy to help out too.

    I've been toying with the idea of proposing Operation: Peer Review for any science/research anons to start compiling critiques of the relevant studies so that we have a resource to hand should any fresh humbug be served up (or indeed any of the past dodgy studies (Spain, Sweden etc) be resurrected).
  3. anonangl Member

    Re: Growl: Flawed Narconon Study Gets Published

    That'd be great Davey Cat
  4. noitulover Member

    Re: Growl: Flawed Narconon Study Gets Published

    Pay to publish? I'd say the rules of academic debate are wide open.

    smart anons are smart!
  5. anonypuffs Member

    Re: Growl: Flawed Narconon Study Gets Published

    /r/ moar
  6. Re: Growl: Flawed Narconon Study Gets Published

    Great - I like the idea of Anon Peer Review. Should we start a blog to post reviews of the relevant bits of 'research'? Or another way of organising things?

    And yes - pay to publish does seem to be changing the rules of academic debate... It will be interesting to see how a journal like that deals with an anon submission.
  7. Davey's Cat Member

  8. Re: Growl: Flawed Narconon Study Gets Published

    Dear Davey's Cat,

    I can has be your personal sex slave?

    K thnx bai!


    Shocking that an organization that abhors psychology would fail at a psychological paper. SHOCKING I SAY!

    (So joining the egghead op. Lurking on it at least...)

    Also, MODS! Get that man a title! (that little bit o text above the avatar).

    I'm thinking: "Internet Supergenius". Which is like a real genius, minus the good standing and credibility.
  9. jargonbot Member

    Re: Growl: Flawed Narconon Study Gets Published

    Adds vote for internets supergenius.

    My apologies in advance for the tl;dr below. Just more questions and wierdness.

    1. About the steaming pile of numbers -- Has anyone found the heavily modified 'CSAP' questionnaire anywhere in the study? The crux of this study come from 19-25 key questions supposedly measuring attitudinal beliefs (lolDarko), while 25 questions measuring attitudes and such were created and added to the survey by Narc. But: “The ability of the intervention to impart knowledge was tested by examining students' ability to correctly answer nineteen items designed to assess assimilation of program content and six questions assessing their ability to apply program messages to drug use decisions and behaviors.”

    Is this right?
    Questions from CSAP
    - Questions about frequency of use for 22 drugs
    - 4 questions about perception from the Student Survey of Risk and Protective Factors (cited as same, but I haven’t read it)
    Questions from Monitoring the Future Survey
    - 12 questions about frequency of use
    - 5 questions about perception
    Questions from Narconon
    - 25 questions about whether students understood what they were being sold (do they have increased recall? Can they apply the tech?).

    2. Is their ‘major finding’ really: after being given repeated drills and Q&As, students tend to answer questions the way they’ve been drilled to answer?

    3. This bothers me: They didn’t discuss the ‘several prevention strategies’ utilized, nor did they describe ‘prevention theory’ at any time. The only mention directly related to their work is ‘social theory’ from Evans 1976 & Evans et al. 1978. There’s a mention of ‘resistance training,’ but the repeated claims about ‘prevention theory’ aren’t backed up or described in any kind of coherent way. And they do a shit job of referencing their peers. Ex: 3 different works authored by Botvin are cited. Two from 1995, and one from 1992. As late as 2007, Botvin has been publishing works about prevention programs and their effectiveness. His recent work is ignored (and it’s excellent), while the work Narc does cite isn’t discussed. They do this over and over throughout the flaming pile (give a 1/2 sentence describing someone else's work, say it's not enough, and then leave it). If a student handed this to me, I would know without a doubt that they wrote their work first and did a lit review for it after.

    4 Notable quotes:
    1 - The aim of this study was to assess the program's ability to change drug use behavior, attitudes and knowledge among youths and evaluate the components of the Narconon drug prevention curriculum against prevention theory.
    2 - The use of the CSAT survey methodology does not make quantifying the reductions in drug use possible and that was not an aim of this evaluation.
    3 - Incorporating several historically successful prevention strategies this curriculum reduced drug use among youths.
    4 - The purpose of this study was to evaluate the capacity of the Narconon drug education program to produce a long-term impact on students' drug use behaviors in a universal (all student) classroom setting.

    tl;dr – 1] Their goal is to find out if the tech reduces drug use. But they can’t know that. So they tested a bunch of kids that said they didn’t use drugs. Therefore, their goal was to find out if the tech reduced drug use -- and it did.


    There are a few people from CAMH in Toronto who work mainly on analyzing prevention methods (and one of them is on the board for this journal...). They're getting this article tomorrow.

    Thanks and many internets to the OP for bringing this and for Mr. Catt's supergeniuses.
  10. XenuLovesU Member

    Re: Growl: Flawed Narconon Study Gets Published

    I am so loving you researchfags :) It gives me much joy to see Narconon get a swift, intellectual kick to the nuts. They've peddled their crazy unchallenged for far too long.

    Again -- great work!

    BTW - credit for the original thread about this study should go to xenubarb, not me. The original thread has long since scrolled off into oblivion:

    From the original thread, it looks like Dave Touretzky caught this study being submitted for review.

    I created an account on and sent them my (simpleton's) observation that I thought the demographics looked skewed... but I don't have the background or credentials to be taken seriously.

    Only dredged this back up because I got an email notification it had been published after the comment period.

    Glad to see you all are on top of this now :)
  11. musketeerwang Member

    Re: Growl: Flawed Narconon Study Gets Published

    "J" on the badscience forums says:

    So selection bias seems to be a HUGE problem for this study. It's like countless pseudo-trials of homoeopathy where they only test people who are willing to sign up for homoeopathic treatment off their own bats. They "want to believe", so give the testers exactly the answers they're looking for. This is what randomisation and blinding are there for, in PROPER scientific testing.
  12. Davey's Cat Member

    Re: Growl: Flawed Narconon Study Gets Published

    The journal's ed copied one of the authors (Ms Cecchini) on his initial reply to me - she's responded, copying me. She has looked at some of the points that I raised and provided unsatisfactory answers. I've thanked her for her message, drawn her attention to the comment I've posted in the journal and invited her to respond in detail.
  13. MongoLloyd Member

    Re: Growl: Flawed Narconon Study Gets Published

    Don't give the editors too much flak..

    I dunno how many of you know how peer-review journals work, but it's like this.. The editors don't review the papers (other than a cursory sanity-check). The review process is external, and it's the reviewer's opinions that really end up deciding whether to publish or not.

    The answers you guys have received are a bit to be expected. Even if a paper is bad (due to lax reviewing for instance), it may be published, because the usual practice is to handle the debate publicly and not via the editorial board. In general you often invite refutations or critique rather than denying publication. (To take an example, the infamous "cold fusion" paper back in the late 80's was published with an editorial noting that they had serious reservations about it - but it was still published)

    While this paper is certainly crap - it's not my field (anyone else here?), so I'm not sure whether or not it's exeptionally crappy for the area. (which is worth bearing in mind).

    Anyway.. publishing in a peer-reviewed paper never meant something was true. (Heck, most papers that are published are *wrong* in some way or another.. that's research.) But it is indeed a step upwards in legitimacy that they don't deserve.
  14. Re: Growl: Flawed Narconon Study Gets Published

    You knows how the Scientologists normally are about copyright. I just noticed that the Substance Abuse Policy article is under a Creative Commons licence:

    You do have to attribute the work correctly, though.

    Of course, it would be very mean if anyone copied bits of the article just in order to take the piss out of this serious science. Oh noes, that would be very mean.

    I wonder, though, is it worth uploading an annotated version of the article to enturbulation at some point?
  15. anonypuffs Member

    Re: Growl: Flawed Narconon Study Gets Published

    /r/ update
  16. TICTpxagain Member

    Re: Growl: Flawed Narconon Study Gets Published

    Good job guys, give em the full logically reasoned hell. Our finest weapon against this cult is reason. We each have our areas of expertise and should put our collective talents where they're most effective. The protests are a public 'face' for us, but the behind the scenes stuff is in my opinion even more important.

    Don't let them get their slippery claws into anything that will allow them to claim credibility.
  17. Re: Growl: Flawed Narconon Study Gets Published

    Firstly, is it worth making a wiki of this paper, to pull it apart line by line?

    At any rate - I've put together a draft comment (to be submitted to the journal anonymously). I've borrowed a bit from here and the bad science forum, but then the comment is to be submitted anonymously.

    If Anonymous has any comments on the comment - or any anons can improve it - then please, do. I know I need to format the Bandolier reference properly, and if anyone has a reference to studies on the negative effects of lying to children about drugs, please let me know. Comment below:

  18. musketeerwang Member

    Re: Growl: Flawed Narconon Study Gets Published

    Really very good, nice work.
  19. Burnt2 Member

    Re: Growl: Flawed Narconon Study Gets Published

    good read; nice work anonresearcher
  20. Mr. Spitzer Member

    Re: Growl: Flawed Narconon Study Gets Published

    There is no rehab that will have an 85% success rate. All narCONon stats are rigged to help convince parents that this is where they should invest their $30k.

  21. Davey's Cat Member

    Re: Growl: Flawed Narconon Study Gets Published

    I've set up a wiki at Welcome - Academics Anonymous to host our collaboration. I've left it open for anonymous contributions / edits at present - we'll see how that works out and move to a registered accounts basis if necessary.

    FYI - still nothing back from the authors of the study that brought all this about, despite my having mailed one of them to give notice of the comment on the site and request a discussion.

    I'm planning to mail her again and copy in the co-author, who appears to be a professional researcher. I'm somewhat surprised at his having gotten involved in this.
  22. Re: Growl: Flawed Narconon Study Gets Published

    The professional scientist looks to be a "gun for hire" if you ask me. He works with a "consulting laboratory" that does scientific studies for cash.

    Cecchini probably shopped around for quite awhile before she found a scientist willing to help her corrorborate the "conclusions" in the study she wanted to do.

    Since that guy is not tied to a university or regular salary, he must generate contracts to pay his various life debts. Hence, it's a symbiotic relationship.
  23. Re: Growl: Flawed Narconon Study Gets Published

    I can has references now. References in code tags, so the URLs are formatted right. Will submit it tomorrow, unless I notice where I screwed up :)

  24. XenuLovesU Member

  25. Himbeertoni Member

    Re: Growl: Flawed Narconon Study Gets Published

    OMG what an awesome thread it took damn long to read but it was totally worth it davey ... amazing absolutely amazing i was visiting your Wiki aswell if there are anymoar academic anons ... join!
  26. Anondelivers Member

    Re: Growl: Flawed Narconon Study Gets Published

    The wiki is a great idea, need to spread it around, I'm sure there are anons willing to contribute.
  27. Growl: Journal rejects my comment

    I've heard back from the Substance Abuse Policy journal now. My comment was rejected for two reasons:

    - because it's "not fair or reasonable" to submit the comment anonymously.
    - because the comment makes references to Scientology. They ask whether I would have the same problems if the programme had come from another religious group.

    Not pleased about the rejection, and I guess I need to decide what to do now. I thought I had explained why I was submitting anonymously: surely the journal must have expected that Scientology's reputation would deter critics from using their real names on replies :(

    Also, fwiw, I do also object when religious groups and others offer unethical education and treatment: for example, while faith-based programmes can be invaluable, I believe that it's important that they're open about the religious aspects of what they do (and most are). The main difference with Scientology is that - while I can criticise most Christian groups without fearing any significant harassment - Scientology's Fair Game policy does change matters :(
  28. musketeerwang Member

    Re: Growl: Flawed Narconon Study Gets Published

    It doesn't matter who the group is, "religious" or not. It's the claims they make.

    What BS. Sorry mate :(
  29. Davey's Cat Member

    Re: Growl: Flawed Narconon Study Gets Published

    Authors' comments now on the article - have only taken a quick look, will look in detail later. Somewhat predictably Ms Cecchini has been reading this thread (Hi Marie! - thanks for stopping by. Stick around and you might learn some interesting stuff).

    I'll go through point by point and respond appropriately.

    Anonresearcher, if you focus just on the soundness of the methodology and supply an appropriate ID then maybe they'll publish.
  30. Re: Growl: Flawed Narconon Study Gets Published

    LOL - does Cecchini want you to declare membership of a forum as a competing interest?

    Thanks for the suggestion. Sadly, I think supplying my name/affiliation(s) would be a bad idea for various reasons, so isn't really an option (due to the hassle it could bring me, not because I'm in the pay of the evil psychiatrists ;) ) Of course, it would be trivial for me to fake an ID, but that would seem unethical.

    I may e-mail and offer to cut down my (anonymous) response, though. It does seem unfortunate if the journal allows the intimidating nature of an organisation associated with the paper to prevent responses: I think I have good reason not to want to be namefagged, and I know others who aren't responding for similar reasons. If the journal don't accept anon comments, this will prevent what looks like basic flaws in the study (using 'more' and 'less' incorrectly), and ethical concerns re a study with minors, from being raised :(

    ETA: btw, unless I'm missing something, the authors don't follow the definition of 'drug' used in their response in the article. Surely more than 21% of students at the start of secondary school will have used drugs - even if just an aspirin or two.
  31. DeathHamster Member

    Re: Growl: Flawed Narconon Study Gets Published

    Lovely to see the hivemind deconstructing one of Narconon's papers that says the Narconon works. Their incestuous net of back-slapping has been looked into before, but with a much smaller pool of people and less skills to draw on.

    They keep cycling this around. FASE does a project, and then ABLE researches the project, and then HeathMed notes ABLE's research, and then Narconon quotes HealthMed's notes, and then FASE records Narconon as confirming their project along with notes from HealthMed, research from ABLE and then they put on their International Detoxification hats and do it all over again...

    Just one big Black Spiral Dance.

    "Science-based" huh?
    [IMG] Church's drug program flunks S.F. test / Panel of experts finds Scientology's Narconon lectures outdated, inaccurate October 2, 2004, Nanette Asimov, San Francisco Chronicle
  32. Davey's Cat Member

    Re: Growl: Flawed Narconon Study Gets Published

    More going on here, but we'll need to be more circumspect in our approach ongoing...

    So, in case you're wondering, this is still in play.
  33. AnonMSW Member

    Re: Growl: Flawed Narconon Study Gets Published

    I find the journal's response to the Anon who sent in a comment disheartening but not unexpected. Please investigate this publication for connections to COS as thouroughly as possible. I am considering "coming out" and writing a response to the journal in my capacity as an academic / mental health pro. Before I finalize any such considerations, I'd like to have a serious understanding of just who the people on the editorial board are and whether or not they have any known links to COS.


    :anon:, MSW
  34. musketeerwang Member

  35. Davey's Cat Member

    Re: Growl: Flawed Narconon Study Gets Published

    AnonMSW - will be PMing you as I'd like your input on the recent developments.
  36. AnonMSW Member

    Re: Growl: Flawed Narconon Study Gets Published

    I welcome all PM's, except from Flag and OSA :)

  37. Davey's Cat Member

    Re: Growl: Flawed Narconon Study Gets Published

    I think it would be of interest to confirm how close to the truth is the following:

    "Marie Cecchini, M.S.
    A molecular biologist, Ms. Cecchini spent a decade working as scientist for Amgen, Inc., developing diagnostic tools and clinical treatments for use in oncology and immunology. Following this, she did doctoral work at the University of Colorado in developmental neuroscience. She established and directed a physical therapy clinic in Boulder, Colorado. In recent years, she worked in the field of drug rehabilitation, at Narconon clinics in Oklahoma and Massachusetts."

    Anyone with any connections at the University of Colorado or Amgen? I'd be intrigued to find out what Ms Cecchini actually did at UoC and where her degrees are from.
  38. DeathHamster Member

  39. AnonMSW Member

    Re: Growl: Flawed Narconon Study Gets Published

    I would like to invite fellow professionals, para-professionals, and peer-counselors to PM me regarding the possibility of establishing a site where such may post semi-peer reviewed (we will be our own peers, hacking and slashing and helping people strengthen their materials) at a new (yet to work the design tasks / etc's out) website where such people may post semi journal-quality articles regarding COS and it's methodology.

    In my imaginary mind's eye, I'd say that it'd be completely legit to post Anonymously provided we list our credentials. People might be disinclined to believe such articles, but if written / edited by those of us with the skills, I think it'll be extremely difficult for that sort of criticism to stick in light of COS' history of "prior bad acts."

    I'm going to post this to Think Tank in the hope that it will generate a response. All people fitting the above profile are invited to PM me, and people with skillsets involving website design are invited to as well. My skillset is limited to behavioral health / science and alas does not include much in the way of web-savvy.

    Thanks for your patience in reading this rather off-topic, yet still I think... relevant post.


Share This Page

Customize Theme Colors


Choose a color via Color picker or click the predefined style names!

Primary Color :

Secondary Color :
Predefined Skins