This lead to some mild twitter outrage amongst the cognescenti of statistical ecology:

Basic and Applied Social Psychology bans both p-values *and* confidence intervals. http://t.co/RkOsChVXpT HT @FidlerF #NoMoreStats

— Michael McCarthy (@mickresearch) February 24, 2015

@bolkerb you can’t directly observe uncertainty, it shouldn’t be reported. @mickresearch @FidlerF

— Dave Harris (@davidjayharris) February 24, 2015

@bolkerb @mickresearch @FidlerF the bane of my life is non-statisticians telling me they know more about f*cking stats than I do.

— Mark Brewer (@BulbousSquidge) February 24, 2015

The editors who wrote the editorial do have good intentions but they just don’t understand the issues.

They start by a firm statement:

From now on, BASP is banning the NHSTP [null hypothesis significance testing procedure].

Which may be going a bit far, but is also a defensible position. After all, p-values are evil, and this is one way of striking a blow for the Forces of Light. The problems come with what the editors do next:

Confidence intervals suffer from an inverse inference problem that is not very different from that suffered by the NHSTP. … Therefore, confidence intervals also are banned from BASP.

Yup. We can’t give estimates of uncertainty. Their argument I ellipsed out above reveals their muddled thinking:

Regarding confidence intervals, the problem is that, for example, a 95% confidence interval does not indicate that the parameter of interest has a 95% probability of being within the interval. Rather, it means merely that if an infinite number of samples were taken and confidence intervals computed, 95% of the confidence intervals would capture the population parameter. Analogous to how the NHSTP fails to provide the probability of the null hypothesis, which is needed to provide a strong case for rejecting it, confidence intervals do not provide a strong case for concluding that the population parameter of interest is likely to be within the stated interval.

This shows that their problems are not with significance tests themselves, but with the frequentist approach to statistics. In the world outside of basic and applied social psychology, a confidence interval does “provide a strong case for concluding that the population parameter of interest is likely to be within the stated interval” if you do the philosophical leg-work.

The key idea which makes frequentist statistics work is the idea that there is a parameter we are interested, say the average IQ of editors of psychology journals. We don’t know what the real value of this is, so we estimate it. The estimate is called the *estimator* (and what we are estimating is the *estimand*). At this point the frequentist can show their art knowledge:

“MagrittePipe” by Image taken from a University of Alabama site, “Approaches to Modernism”: [1]. Licensed under Fair use via Wikipedia.

The estimator is not the estimand, but hopefully it is a good estimator of it, and one can make probability statements about the parameter (such giving a probability that it is less than a certain value), at the cost of making the probability being of the data. In the frequentist interpretation, what we see is the data, which is random, so we can only make statements about that. Statistics, such as parameter estimates, are functions of the data, so statements about variability of the estimators have to be statements about variability of the data.

If one accepts the frequentist interpretation, then you have to accept that all your summaries are statements about the data, not the parameters. As most analyses are frequentist, the editors are ditching these analyses. This gets worse, as we’ll see.

The editors then go close to throwing the main alternative under the bus too:

Bayesian procedures are more interesting. The usual problem with Bayesian procedures is that they depend on some sort of Laplacian assumption to generate numbers where none exist. The Laplacian assumption is that when in a state of ignorance, the researcher should assign an equal probability to each possibility. … [W]ith respect to Bayesian procedures, we reserve the right to make case-by-case judgments, and thus Bayesian procedures are neither required nor banned from BASP.

The Laplacian assumption? No, Bayesian methods *don’t* depend on it. Indeed, any good Bayesian know that it can’t depend on it, because there is often no unique way of assigning equal probability, for example with variances one could assign such a prior to the variance, the standard deviation, the precision (1/variance), or the log of the variance. In reality, Bayesian methods rely on the assumption that a prior distribution represents the knowledge of the parameter(s) before the data are seen. It’s not clear what would happen if someone submitted a paper with informative priors, as the editors seem to think that this isn’t possible.

If we’re not allowed to be Frequentist, and only be Bayesian if we’re persuasive, what’s left? This:

However, BASP will require strong descriptive statistics, including effect sizes.

Wonderful! We can describe our data, and we can give effect sizes. But (a) we are not allowed to give measures of uncertainty around the effect sizes (OK, standard errors don’t look to be banned, but if confidence intervals say nothing, then surely standard errors, from which confidence intervals are often derived, must be equally suspect), and (b) these are purely descriptions of the data. If we are to assume they are something more, i.e that we are measuring something that can be extrapolated, then we have to assume that the descriptive statistics are measures of an underlying parameter. But by the same logic as above, if a confidence interval says nothing about the range of likely values of a parameter, then a point estimate (i.e. a descriptive statistic) says nothing about the best value. In other words we can say that the average IQ of the editors of psychology journals that we tested is 93 , but the logic of the BASP editors implies that we can’t use this to say anything about editors of psychology journals in general.

So what’s a social psychologist (whether basic or applied) to do? If it’s enough to just describe your data, you’re fine. But what if, say, you did an experiment? You then want to make inferences, but the BASP editors don’t like them. You do have a few choices, which all rely on varying degrees of bluffing:

- provide standard errors but not confidence intervals. Everyone will multiply by 2 to get their own confidence intervals anyway,
- be Bayesian, with informative priors and try to persuade the editors that this is OK.
- claim you’re using fiducial probability. The only person to understand this was R.A. Fisher, but that’s OK. You can simply cite one of his papers as a justification. The editors won’t underrstand the issues, so their decision will be just as arbitrary as it would be anyway.

If you try option 3, please tell me the results.

]]>One reason for this failure is that the prize is given out to too many people at once. So the news becomes “big prizes given out”, rather than “X wins big prize”. If the point is to honour scientists, then the format shouldn’t hide them. The use of celebrity doesn’t help either, as that becomes the news focus. The BBC’s coverage takes 6 paragraphs to mention any of the winners, with only 4 paragraphs given over to the prize recipients (it ends with Athene Donald being rather critical. Was she coached by Dr. Aust?). The Guardian takes 7 paragraphs to mention the winners, but at least it lists the winners at the end of the piece.

Of course there are real rock stars of science, but I’m not sure the Breakthrough Prize really helps to showcase them. Partly because real rock stars look like this…

(spot the physicist on hiatus)

… and they don’t become rock stars by winning, say, the Mercury Prize. They get there through repeatedly being in the public eye with their music: that’s why they are rock stars, and not one hit wonders.

Rock stars also get that way by entertaining the public (which is, after all, their job). For scientists, too, their ability to attract the public eye is part of their rock star status. This is why Richard Feynman, say, is so well known. Being given a prize and then standing around like a penguin saying how wonderful it is doesn’t help, especially when you’ve had Benedict Cumberbatch doing the same thing earlier in the report.

Think I’m just being grumpy? Well, here’s a simple of whether the Breakthrough Prize has helped raise the profile of the winners: ask yourself how many of last year’s winners you can name without looking them up. And then how many of this year’s winners.

]]>The ICMJE recommends that authorship be based on the following 4 criteria:

Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND

Drafting the work or revising it critically for important intellectual content; AND

Final approval of the version to be published; AND

Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

and anyone who doesn’t meet all of those criteria shouldn’t be an authors, but should be acknowledge. Goldacre, bless his little cotton socks, points out that this means that many people who contributed (e.g. those who ran the trials, and the people who wrote the paper – in other words the people from the bad pharma company) aren’t credited as authors. Which is true, but he ignores the other side of the coin. A couple of pages before discussing the ICMJE guidelines he quotes an academic, Dr. Lisse, who told the New York Times

“Merck designed the trial, paid for the trial, ran the trial,” Dr. Lisse said. “Merck came to me after the study was completed and said, ‘We want your help to work on the paper.’ The initial paper was written at Merck, and then it was sent to me for editing.” … “Basically, I went with the cardiovascular data that was presented to me,”

Now, I’m not sure this qualifies Dr. Lisse to be an author under the guidelines, so perhaps he should be removed as an author. Which then raises an interesting idea. What if none of the authors fulfils the criteria? Could we then see the authorless paper?

]]>The big day arrived when the tower was to be brought down arrived on Sunday. There was a nifty animation that showed how it would be done: first blow off the outside, and then bring down the main part of the tower, in two pieces.

http://youtu.be/A_SWkNYjE2c

The lower piece was aimed right at our building (my office is on the other side; the labs would take most of the damage of the bottom half of 32 floors of brutalist architecture falling on top of it), so we had lots of preparation to do: wrap half of the building, turn almost everything off (our servers are in the main Senckenberg building, which is only about 100m up the road), and make sure nobody is in the building on Sunday morning. On Sunday morning I was 4.8km away, as the google maps, streaming the live coverage from Hessischer Rundfunk over the web. All went well – we even heard the sound live, a couple of seconds after the building went down. Although all of the humans were outside an exclusion area, the Senckenberg’s T. rex was allowed to stay inside (would you argue with him?), so he managed to catch this footage from a camera that seems to be attached to his left leg:

http://www.youtube.com/watch?v=Zwdb4wa2qJQ

Sunday afternoon was busy for some people, running around checking that everything was OK. Our building was intact – there wasn’t even any dust in my office, even though there was a lot outside a couple of hundred metres away:

And this is the remains of the AfE tower:

Now we just have to suffer the sounds of this pile of wreckage being taken away. And then they’ll probably insist on fiulling the space with a new post-Brutalist tower.

]]>http://youtu.be/geNDLwNdZOA

And not content with that, they’re also hosting Paperworld and Creativeworld this week too. So we can look forward to a full week of fun festive origami!

Unfortunately we have to wait until October for Cleanzone to get rid of the mess.

]]>We’ve got an EXCITING(!) EXHILARATING(!!) ENTERTAINING(!!!) EXASPERATING(!V) opportunity for someone wanting to do a post-doc in Frankfurt, working in the Data and Modelling group here at BiK-F. I’ll be one of the people supervising the project. The official announcement is here (pdf), and below. Although it’s initially for about 15 months, there’s a reasonable chance of getting an extension, depending on how things go.

Of course, I think Frankfurt is a great place to work. BiK-F is an institute connected to the Senckenberg museum, and the Goethe university. The project itself is part of a German project to create a biodiversity database for German research: this part is to show that such a database would actually be useful.

The full advert follows. Feel free to ask about this in the comments below.

The Biodiversity and Climate Research Centre (BiK-F) has been founded by the Senckenberg Gesellschaft für Naturforschung, the Goethe-University Frankfurt am Main, and additional partners. It is funded by the Federal State of Hessen through its Initiative for the Development of Scientific and Economic Excellence (LOEWE). The mission of the centre is to carry out internationally outstanding research on the interactions of biodiversity and climate change at the organism level.

The Project Area E “Data and Modelling Centre” invites applications for a Postdoctoral Researcher

**“Statistical modelling of species distributions” (Ref. #E41)**

Your tasks:

- Improve our tools for estimating species distributions
- Statistically relate species occurrence data and environmental layers
- Handling of remote sensing products related to habitat quality (e.g. biomass, area burned, tree

cover) - Sampling bias analyses
- Develop toolboxes for a DFG-funded biodiversity data centre (visualization and analyses)
- Work in an interdisciplinary team with informatics and GIS experts and bio- and geoscientists

Your profile:

- PhD degree in Ecology, Mathematics, Statistics, Bioinformatics, Geography, Environmental Science

or a related field - Strong expertise in statistics and numerical modelling
- Advanced skills in analysis of large datasets and/or ecological modeling
- Special expertise in Bayesian species distribution modelling would be an advantage
- Strong track record of international publications in peer-reviewed journals
- Excellent written and oral communication skills
- Interest to work in interdisciplinary teams

Salary and benefits are according to a public service position in Germany (TV-H E13, 100%). The contract shall start as soon as possible and will be initially restricted to May 2015. An extension is intended being subject to personal performance and availability of funds. The Senckenberg Research Institute supports equal opportunity of men and women and therefore strongly invites women to apply. Equally qualified handicapped applicants will be given preference. The duty station will be Frankfurt am Main, Germany, but exceptions might be possible. The employer is the Senckenberg Gesellschaft für Naturforschung.

Please send your application before February, 15th 2014 preferred by e-mail (attachment in a single pdf document), mentioning the reference of this position (Ref. #E41) and including a letter outlining your suitability for the post, a detailed CV, contact details of 2 references and a list of publications and funding to the Administrative Director:

Herrn Dr. Johannes Heilmann

c/o Senckenberg Gesellschaft für Naturforschung

Senckenberganlage 25

60325 Frankfurt

E-Mail: recruiting@senckenberg.de

For scientific enquiries please get in contact with Prof. Dr. Thomas Hickler, thomas.hickler@senckenberg.de.

Further information: http://www.bik-f.de.

**EDIT: and if you don’t want to work with me, you could apply to work with Tord Snäll in Sweden, instead**.

Dear Colleague,

RESEARCH ARTICLES PUBLICATION – NO PUBLICATION CHARGE

Pinnacle Journal Publishes peer-reviewed, open access journals covering a wide range of academic disciplines.

We invite you to submit your research paper for publication in PINNACLE JOURNALS. Send your paper via e-mail attachment only to: submission.pjpub@gmail.com.

Regards,

Publisher

Pinnacle Journal Publication

Note: To opt-out from our email list, simply send a blank message with STOP as subject.

Impressive, no? Not even a web page. This may have come from Pinnacle Journal Publication, but the email came from a wildblue.net domain (which is, apparently, a US internet provider), and the gmail submission address just shouts professionalism, doesn’t it?

Ah well, at least it amused me for a few minutes.

]]>Old Methods

(which, alas does not seem to give the full Soundcloud plugin experience on this page. Poo.)

and check the MEE blog post for a list of who said what.

You will notice I evaded interview. After considerable though, I decided that the oldest method I’ve used is procrastination, something even older than regression. My newest method is clearly twitter, and I’d really like a method that would let the computer do all the hard work of making my MCMC work, but not too quickly (because MCMC runs are, in effect, more modern implementations of procrastination).

What method do *you* use?

Now, it’s little known but a few years ago GrrlScientist actually experimented with breeding attack budgies. They sound all cute and harmless, and indeed on their own they are. But it is as a group they are the menace Grrl requires for her work.

httpvh://www.youtube.com/watch?v=B8ZQjelJ3zA

Imagine if she has decided to bring down the wrath of her attack budgies upon you. First one or two arrive – they’re cute, and might nibble but they sound nice and get up to jolly japes. Then a few more arrive. More cuteness, until they start exploring. Soon a couple of keys on your laptop are removed, or they’ve knocked over your drink. Then as more descend the noise level grows they pile onto your houseplants and furniture. Into the kitchen to raid the fridge and cupboards. By now there is cacophony and riot: anything not solid enough is being chewed into pieces by many little beaks. The wallpaper is ruined, as is the carpet: if it hasn’t been chewed, it’s accumulating small parrot droppings.

Eventually there is some unheard signal, and the budgerigars start to drift away. They leave in groups, still constantly chattering. Perhaps about the fun time they’ve had, or possibly discussing their next target. As they depart, you can examine the damage – the wrecked rooms, chewed to pieces and stained white. The food partly consumed, but mainly thrown around with the sort of wastefulness that would put even humans to shame. A few feathers still drift in the air as you realise the folly of crossing swords with a master parrot breeder.

That was the plan and it was very effective. But it was aborted after the first attempt to get all the buggers back into their cages.

]]>In there paper, Evans and a plethora of co-author present this argument:

[W]e argue that there is usually a trade-off between simplicity and generality, such that simpler models are, all other things being equal, less general than are complex models. For example, a nonlinear population growth equation such as

dN/dt=αN+βX^{1+a}represents a large family of models, the members of which correspond to the constant parametersαandβbeing set to particular values (whereasacan take any value). Ifβis set to zero, we obtain a simpler linear equation,dN/dt=αN. Obviously, the nonlinear equation includes the linear one as a special case. Thus, the more complex equation represents a larger family of models than the linear ones and, therefore, is more general. It can pick out all the real systems that are described by the linear equation plus a range of others.

(I have slightly changed the equation from what the authors presented, to make my argument clearer. I think the original argument is unchanged, though)

i.e. simple models are less general because they cover less of the model space. Which is odd, because I think this is precisely why a simpler model is more general.

The argument for the greater generality of a simple model is that any mechanism not in the model is not relevant, so the conclusions we get from the model are relevant whatever other mechanisms might be acting. In other words, if we are using the simple model *dN/dt* = *αN* we are assuming other models like *dN/dt* = *αN* + *βX*^{1+a}, *dN/dt* = *αN* + *βe*^{Y}, *dN/dt* = *α sin θ _{t} N* etc. etc. will show the same behaviour that we are investigating. Of course, they will differ in other ways but the purpose of any model should be to investigate specific phenomena, and how it behaves outside of that is generally not relevant.

On this reading, a more complicated model is less general because it fixes more of the model: *dN/dt* = *αN* + *βX*^{1+a} precludes terms like *e ^{δX}*. Less abstractly, if we have built a model with sexual reproduction, we should not be using it to make inferences about asexual populations.

Of course, we can expand a model to have proportions of sexual and asexual reproduction, which would be more general than one that only assumed one mode of reproduction. But in practice, I think more complex models are usually developed in a different direction: they are made more complex by adding more mechanisms (e.g. age-structured reproduction, individual-based behaviours). These end up being much less general: a model of the behaviour of Cromerian giraffes will work well to predict their behaviours (such as cycling), but will not be of much use for predicting the head-perching behaviours of parrots.

Evans *et al.* suggest that complex models can be made general by exploring their behaviour over a range of parameters:

The price for complexity is that such models usually need to be tied to data from specific systems. We are still left with the problem of how to generate general insights from models that are tied to specific systems. A key strategy currently used to solve this problem is the use of simulation experiments. Such experiments are performed on models, but parallel the kinds of experiment performed on laboratory systems. Techniques include analysing confounding factors, such as heterogeneities and stochasticity, changing the number of types of entity and process considered in the system, and systematically varying the parameters and variables of the model to determine whether its predictions are strongly or weakly influenced by changing values and, thus, which processes are more or less important dynamically. Utilising such simulation experiments requires the systematic consideration of possible (but not actually occurring) scenarios to understand the scope and limits of the model in question.

In tying a complex model to specific systems, a modeller is making assumptions specific for that system, which makes it less general (e.g that giraffes cannot fly, or unicycle beyond certain points). Exploring the (large) parameter space still only explores the model within these assumptions. In practice the lack of generality is even worse, because with more parameters it is more difficult to explore the parameter space: with a simple model like *dN/dt* = *αN* we can fully characterise its behaviour over all possible values of *α*. But with (say) 20 parameters and a model too complex for an analytic analysis, some choices have to be made about which parameters to explore, and what parameter values to use. We simply don’t have the computing power to explore the full space of parameters. Thus any generality that is gained by adding complexity is lessened simply because we are human, and cannot understand the model across the full range of its generality.

The other problem with using complex, specific, models to explore general behaviours is that it becomes difficult to understand why a model is behaving the way it is. Modellers are reduced to carrying out experiments with specific choices of parameters, and then using statistical methods like ANOVA to summarise the behaviour of the model. In essence it is treated as a black box. But the point of a model of general phenomena is to understand the systems where we see the phenomena: to unpack the black box. I suspect that the complexity of models is used as an alternative to thinking: a model is built to predict the behaviour of Cromerian giraffes, and subsequently the modeller thinks that the model can be used to explore general locomotory behaviours, without the bother of thinking about what aspects of locomotion (unicycling, flying, attraction to balding heads etc.) are the most relevant. Thinking first, and then developing a minimal model should be a preferable path, as it removes unnecessary parts of the model. this makes it easier to focus on what is important and thus easier to understand what the model is telling you. It also means there is a smaller parameter space to explore, and if you are simulating, each simulation will usually take less time (because there are fewer functions to evaluate). What’s not to like? Other than having to think, of course.

One strange idea that crops up in this paper is this:

[Simple models] do not need to be tested against specific data because they represent concepts rather than systems.

and

Unlike in physics, where general models have to make testable predictions, ecology has embraced an approach where models claiming generality are untestable in any real system

Frankly, this idea is rubbish. Evans *et al.* criticize Volterra for comparing his predator-prey models (which produce cyclic dynamics) to data on fisheries in the Adriatic (where the data have cycles) for being confused. Likewise:

May and Anderson in 1979 compared the output of their model with data from real populations exposed to diseases. They found that ‘some of the theoretical conclusions can be pleasingly supported by hard data, while others remain more speculative’. This confusion of model purposes gives the false impression that simple demonstration models can provide actual explanations of specific systems.

But no explanation for the confusion is given – apparently it’s just that Volterra, Anderson, and May all erred by not doing things the Evans *et al.* way. Even if simple models “show that the modelled principles are sufficient to produce the phenomenon of interest” (which is a reasonable point of view), they are still intended as explanations of the real world. So surely they have to be tested against the real world, one way or another. To deny that is to deny that simple models of real-world phenomena are models of the real world. Idealised, yes, but they are still meant to say something about the real world.

I think the test of a simple, general, models could be rather informal. Volterra showed that predator-prey dynamics can produce cycles. One does not need to fit his model to the data to say that it works as an explanation: it is enough to see cycles in data with predators and prey. At Intecol last month, Sarah Calba gave a presentation about “prediction”, and argued that the concept is multi-faceted. We predict about the future (e.g. what the distribution of Cromerian giraffe will be in 2063), but we also make different sorts of predictions about the past and present, e.g that Cromerian giraffe cycle. And this can be tested, by going to Cromer and looking for giraffe, to see if any are cycling. The predictions are more qualitative, and serve a different purpose: they let us test our model explanations of the natural world, rather than tell us about the state of system that we are predicting for.

The beauty of simple models is that they help us understand some aspects of a range of real world systems. Volterra built his model to explain cycles in Adriatic fish. But the same model also provides an explanation of cycles in North American lynx. And chemical reactions. And it is also used in economics. But apparently it’s not general.

Evans MR, Grimm V, Johst K, Knuuttila T, de Langhe R, Lessells CM, Merz M, O’Malley MA, Orzack SH, Weisberg M, Wilkinson DJ, Wolkenhauer O, & Benton TG (2013). Do simple models lead to generality in ecology? Trends in ecology & evolution DOI: 10.1016/j.tree.2013.05.022

]]>