Share Interesting Content/Stats Papers

[trinker]

I am starting this thread because I have often seen people share papers or references for interesting papers they've been reading. This is a more formal version so we have a record we can go back to.

I was envisioning something like we share:

• The reference
• A small sentence ditty about it
• Optionally a link
• Optionally a question or thoughts

I thought this may open up dialogue, expose us to stats in other contexts and build a sense of community as we share work relevant to our interests.

 

[trinker]

I look at cognitive load and multimodality a bit. I heard about this paper on the radio last night about eye blinking when our cognitive load is challenged.

Vredeveldt,A., Hitch , G. J., & Baddeley, A. D. (2011). Eyeclosure helps memory by reducing cognitive load and enhancing visualisation. Psychonomic Society 39, 1253 – 1263, doi 10.3758/s13421-011-0098-8

http://annelies.vredeveldt.com/wp-c...ognitive-load-and-enhancing-visualisation.pdf

 

[TheEcologist]

I've mentioned this one before, but I recently read the paper by Johnson.

He makes the case that significance determined with classical test in fact only amounts to marginal evidence in a Bayesian setting. He goes further to state that the wide use of classical tests - were alpha levels of 0.05 - 0.01 are seen as significant - is likely major contributor to the appallingly large proportion of studies that fail to reproduce results.

He argues that the application of Bayesian uniform most powerful test (some developed by him) would reduce this proportion greatly.

I suspect that a major contribution is also that many classical tests are being applied to study designs that do not correspond to the design of the test. Hence I am highly skeptical that 'standard' Bayesian tests built for the same ANOVA type study designs will solve this.

I wonder what the rest here thinks of this work?

 

[Jake]

@TheEcologist, do you think you could upload a PDF of the Johnson paper to our Talkstats Dropbox?

 

[Mean Joe]

He makes the case that significance determined with classical test in fact only amounts to marginal evidence in a Bayesian setting. He goes further to state that the wide use of classical tests - were alpha levels of 0.05 - 0.01 are seen as significant - is likely major contributor to the appallingly large proportion of studies that fail to reproduce results.

Is the paper online to read? I think without needing to go fully(?) into a Bayesian setting, we can see why conclusions from studies with many tests are so often wrong. Does that paper make his case around something like microarray studies?

"In [Ioannidis'] example, he supposes that 100,000 gene polymorphisms are being tested for association with schizophrenia. If 10 polymorphisms truly are associated with schizophrenia, the pre-study probability that a given gene is associated is 0.0001. If a study has 60% power (β = 0.4) and significance level α = 0.05, the post-study probability that a polymorphism determined to be associated really is associated is 0.0012. There’s a 99.8% chance that the result is false."

 

[noetsi]

I have a bunch of academic papers related more to what I think of as practical issues (to me anyhow) - say limitations on test for stationarity and solutions. I am not sure if there is any interest in that type of paper here.

 

[TheEcologist]

@TheEcologist, do you think you could upload a PDF of the Johnson paper to our Talkstats Dropbox?

I accessed it directly so it doesn't seem to be behind a paywall. Please check if you can, and otherwise... who will send me the link for the TS dropbox folder?

 

[Jake]

Oh, yeah you're right, I can access it directly, just found the link (for others looking, it is the tab with the red Adobe logo).

 

[TheEcologist]

Is the paper online to read? I think without needing to go fully(?) into a Bayesian setting, we can see why conclusions from studies with many tests are so often wrong. Does that paper make his case around something like microarray studies?

"In [Ioannidis'] example, he supposes that 100,000 gene polymorphisms are being tested for association with schizophrenia. If 10 polymorphisms truly are associated with schizophrenia, the pre-study probability that a given gene is associated is 0.0001. If a study has 60% power (β = 0.4) and significance level α = 0.05, the post-study probability that a polymorphism determined to be associated really is associated is 0.0012. There’s a 99.8% chance that the result is false."

That is an incredible false rate. However, is that related to not understanding the statistics or not having the correct tools to study the phenomenon? Using the wrong statistical test cause you don't understand the test, is one thing but if your using the wrong methods to derive data no statistical test is truly suited.

Johnson works with a dataset of almost 1000 tests done in psychology.

 

[Dason]

I should point out that in microarray (and RNA-seq) experiments typically measures are taken to control the false discovery rate. http://en.wikipedia.org/wiki/False_discovery_rate

 

[gianmarco]

I look at cognitive load and multimodality a bit.

@Trinker:
I am going to publish an article in which, for the purposes of my study, I review some literature on cognitive limits in groups decision-making and on cognitive constrains to human group size. Just in case someone could be interested...I will post the link as soon as it will be available from PLOS ONE.
By the way, nice idea that of this thread. I believe that it would be nice/useful to implement a way to sort the topics in order to make them easily searchable by users. I do not know if this could be actually done.

Gm

 

[Mean Joe]

That is an incredible false rate. However, is that related to not understanding the statistics or not having the correct tools to study the phenomenon?

My reading of it, if you rely on alpha=0.05 to get "significant results", then you are being lazy and not asking the necessary questions ("What is the probability that the results are correct?").

Following the calculations, if you use a conservative Bonferroni-corrected alpha (and keep beta constant) then the probability of a false result can get <1%.

Anyways. I'll read that Johnson paper some more. You don't need to go Bayesian to see that alpha is not one size fits all, especially with nowadays studies.

 

[TheEcologist]

I've found another interesting study, by Nakagawa and Schielzeth. They propose a method to calculate R2 values for mixed effect models and conclude "The proposed method has the potential to facilitate the presentation of R2 for a wide range of circumstances".

This strikes me as silly, and I would wonder what other think of this, and the usefulness of R2 values in a mixed model case.

 

[ted00]

This strikes me as silly

As someone who's had to fit many nonlinear mixed-effects models, usually for the purpose of prediction, I agree with you. I didn't look at that paper yet, but I've seen others that desire to boil it down to one number, a R2-like value. For prediction models, cross-validation repeated many times (not split the data into 1 fitting and 1 compar data set, but do many times) is better; for model building, model comparison, hypothesis testing -- LRT, BIC, etc..

 

[bugman]

Trinker, I thought you might like this one...

Sorry, I dont have access, but it looks pretty interestesting:

http://www.cell.com/trends/ecology-evolution/abstract/S0169-5347(14)00016-0

 

[trinker]

Thanks. Looks interesting and applicable. It's not easily accessed so I put in for it via my Library's REQUEST. I'll share with you if you'd like Bugman.

 

[bugman]

That would be great!