Experimental economics has considerably developed over the last 70 years. However, actual experimental practices, as well as practices that are considered as appropriate, have considerably changed over years. Even though not for the same reasons depending on the period, lab experiments have always been subject to vivid debate.
Two main periods of debate can be identified. The first (resp. second) period covers the 1990' (resp. 2010') and coincides with the rapid growth of experimental research in the lab (resp. field). Our claim is that, beyond apparent differences in arguments and stakes between the two periods, deep similarities actually exist across first- and second-period methodological debates that make history seem to repeat again.
First, arguments that are used in the present debate to disqualify lab experimental research appear to be roughly the same as those used in the 1990'; their common ground is to claim that lab experimental methodology is too flawed to produce reliable results. Even though in both periods criticism was based on lab experiments' lack of both internal and external validity, external validity appears to be the primary methodological concern. In this respect, the main difference between the first- and second-period arguments is that, while in the first period there was some hope for conducting better (that is, more externally valid) lab experiments provided the methodology was improved, in the second (current) period, criticism against lab experiments is more radical: they by definition cannot offer good external validity – contrary to field experiments.
Second, in both periods, the methodological debate is actually (though not officially of course), and quite interestingly, the expression of a similar theoretical opposition between the defenders of standard theory (who tend to strongly criticize lab experiments) and the advocates of challenging theories (who rather tend to promote lab experiments). It is worth noticing that, in the first period, the methodological debate rather involved the level of rationality of behaviour (with less rational behaviour observed in the lab than assumed by standard theory), while in the second (current) period, the focus has moved to the level of self-interest (with more altruistic behaviour observed in the lab than assumed by standard theory). Since standard theory is based on the combination of perfect rationality and self-interest, it is obvious that the empirical weakening of either principle may lead to severely undermine the status of standard theory (e.g. EU theory as regards individual decision making under risk, Nash equilibrium in game theory). This may easily explain why its toughest defenders may be tempted to discredit experimental results for actually theoretical rather than methodological reasons.
So the main point we wish to make here is that methodological issues, though important per se, are often raised for, and obfuscated by, some hidden stakes that should be identified before deciding which pieces of criticism raise a real problem and which are just a pretext. These hidden stakes also contribute to explain why methodological positions may be so entrenched. Moreover, methodological discussions may obviously be harmed, and maybe perverted, when stakes at play are actually not only, and even not mostly, methodological. This is the case with immunizing strategies that aim at defending at all costs a theory that has been repeatedly invalidated in the lab.
In this contribution, we will describe the main arguments and counter-arguments used in the 1990' (resp. 2010') debate to try to identify how non-methodological stakes enter and interfere in supposedly methodologically-oriented exchanges between scientists.