This is analysis of source data in the online db, which is basically the same as when it was first collected. This is clearly dated, so pulling the data and saving as a data frame for reproducibility.
g <- graph_heroku_graphene(remote = "heroku")
query <- "MATCH (o:Outcome)<--(n:Source)-->(m:Input) RETURN n.url AS url, n.title AS title, m.name AS input, o.name AS outcome, n.change_of_effect AS effect, n.trial_design AS design;"
source_data <- cypher(g, query) %>% tbl_df
devtools::use_data(source_data, overwrite = TRUE)The one thing we can use for the time being to do meta-analysis is “change of effect”. We’ll have to change that, but for now let’s see what we got.
library(nooseed)
data(source_data)
summary(source_data)## url title input
## Length:339 Length:339 Length:339
## Class :character Class :character Class :character
## Mode :character Mode :character Mode :character
## outcome effect design
## Length:339 Length:339 Length:339
## Class :character Class :character Class :character
## Mode :character Mode :character Mode :characterWhat is the distribution of effects?
source_data %$%
effect %>%
as.factor %>%
summary## Both Decrease Increase None
## 2 48 171 118What outcomes have a “decrease” effect?
source_data %>%
filter(effect == "Decrease") %>%
select(input, outcome) %>%
unique %>%
mutate(outcome = as.factor(outcome)) %>%
select(outcome) %>%
table## .
## Cognitive Decline Irritability Memory
## 14 4 1
## Relaxation Stress Subjective Well-Being
## 1 11 2Cognitive decline, irritability, and stress are outcomes where we want to see a decrease effect. For memory, relaxation, and subjective well-being, it is odd that these are reported as having a decrease effect. I expect to see weird things like this with correlational studies (observational designs, cohort studies), but not with randomized trials, where randomization controls for confounding factors, and where the hypothesis ought to be one-sided. Let me see what types of designs these are.
source_data %>%
select(url, design) %>%
unique %>%
mutate(design = as.factor(design)) %>%
select(design) %>%
table## .
## Cohort Double blind
## 39 192
## Meta analysis Randomized trial
## 1 2
## Uncontrolled / observational study
## 1I am not sure what the difference between a double-blind and a randomized trial is, I supposed they are the same except the latter is not double-blinded. Why even do that? Let’s read them.
source_data %>%
filter(design == "Randomized trial") %>%
select(url) %>%
unique## # A tibble: 2 x 1
## url
## <chr>
## 1 http://www.ncbi.nlm.nih.gov/pubmed/24958525
## 2 http://www.ncbi.nlm.nih.gov/pubmed/26305649Upon reading these two papers, it turns out they were indeed double-blinded trials. Must have been some change in how the data was originally recorded.
How about the cases where memory, relaxation, and subjective well-being are decreased?
source_data %>%
filter(outcome %in% c("Memory", "Relaxation", "Subjective Well-Being"),
effect == "Decrease", design %in% c("Double blind", "Randomized trial")) %>%
select(url, input, effect, outcome) %>%
unique## # A tibble: 3 x 4
## url input effect
## <chr> <chr> <chr>
## 1 http://www.ncbi.nlm.nih.gov/pubmed/16541243 Caffeine Decrease
## 2 http://www.ncbi.nlm.nih.gov/pubmed/16541243 Caffeine Decrease
## 3 http://www.ncbi.nlm.nih.gov/pubmed/1324250 Marijuana Decrease
## # ... with 1 more variables: outcome <chr>The first article is Subjective, behavioral, and physiological effects of acute caffeine in light, nondependent caffeine users. Here is what it had to say about the proported decrease in memory:
Caffeine (450 mg) significantly decreased the number of digits remembered in the backwards Digit Span task showing an impairment of working memory. Caffeine did not affect digits remembered in the forward series. Caffeine (150 and 450 mg) significantly increased the number of hits in the Visual Vigilance task, and the 450-mg dose significantly decreased reaction times Fig. 6. Caffeine did not alter the number of false positives at any dose. Caffeine did not affect performance upon the DSST and Stop Task.
Firstly, it appears there was an error in the recording of the outcome – this was an evaluation of working memory, not memory.
Secondly, 450 mg of caffeine is high, (on average a cup of coffee is about 100 mg), so this could be something like an overdose. But the decrease is a trend that gets worse in increasing doses (see Fig. 6 in the article). Moreover, they compare to a baseline of performance before caffeine intake (if anything you would expect results to improve the second time as a result of learning). I might need to re-evaluate my idea that hypotheses of effects on cognition are all one-sided.
In contrast, the report of caffeine’s decreasing effect on subjective well-being appears, (ahem), more subjective:
Caffeine did not affect the MBG (euphoria, Figure 1) scale but the 150 and 450 mg doses significantly increased ratings on the LSD (dysphoria) scale… In contrast to the effects of caffeine, d-amphetamine produced significant euphoria relative to placebo (MBG scale, Fig. 1) but there were no significant dysphoric feelings (LSD scale) or decreases in sedation (PCAG scale)… There were no significant effects of caffeine upon DEQ ratings of “like drug” or “want more”. In comparison, d-amphetamine produced increases in “feel drug” that appeared similar in magnitude to peak effects of caffeine (Fig. 2)… Unlike caffeine, 20 mg d-amphetamine significantly increased ratings of “like drug” and “want more” relative to placebo. Caffeine produced a significant increase in … “positive mood” (data not shown)… D-Amphetamine also increased ratings on the POMS “vigor” and “arousal” scales and peak effects appeared larger than those induced by caffeine.
The comparison to d-amphetamine was from a different study, so while interesting, drawing statistical conclusions in this context is highly questionable. Also, it seems subjective reporting of “good feelings” is higher for normal doses, and only lower than baseline for the 450 mg. Not surprising, most people would feel weird after having 4.5 cups of coffee back-to-back. Since that is not the ideal dose, the “Decrease” assessment seems inappropriate.
The second article reporting that marijuana decreases relaxation was Regional Cerebral Blood Flow After Marijuana Smoking. The article says:
Tension and anger (as measured by the Profile of Mood States) showed significant increases after marijuana but not after the placebo (drug/placebo by time; tension, F = 2.38, P < 0.032; anger, F = 2.42, p < 0.029). Figure 4 presents the results of analysis of ratings of intoxication. A significant drug/placebo-by-time effect was indicated (F = 18.68, P “’” 0.001). The ratings changed in a pattern consistent with dose of THC, but there was little difference between the two doses.
I don’t know what this “Profile of mood states” is, but the F-test statistics look sound. So marijuana makes you “angry”? What do other studies say about marijuana?
source_data %>%
filter(input == "Marijuana") %>%
select(outcome, effect, design) %>%
unique## # A tibble: 7 x 3
## outcome effect design
## <chr> <chr> <chr>
## 1 Attention None Double blind
## 2 Cognition None Double blind
## 3 Memory None Double blind
## 4 Working Memory None Double blind
## 5 Irritability None Cohort
## 6 Irritability None Double blind
## 7 Relaxation Decrease Double blindLooks like there isn’t much going on with marijuana anyway. It makes sense that until what was recently an illegal street drug would be under-studied. Probably would have similar results for LSD and others.