In their tribute to Gary Becker, Andrei Shleifer and Edward Glaeser document the sources of early skepticism of non-rational modeling of economic behavior:
In his brilliant 1962 essay “Irrational Behavior and Economic Theory,” Becker pointed out that economics could still yield meaningful predictions if humans acted in a random fashion but were subjected to resource constraints. Still, Becker insisted on the assumption of human rationality, despite his extremely broad view of what enters human utility. Perhaps he feared that opening the doors to irrationality would create a discipline without order, nor the ability to generate falsifiable predictions. Only in recent years, with tremendous growth of experimental and survey evidence, have economists been able to move away from rationality and still generate testable and falsifiable predictions.
A related view that continues to resonate among many academics is that "behavioral economics lacks a unified model" and that it is a just a collection of examples when judgments are skewed by biases. Many critiques of behavioral economics seem to be centered on the suspicion is that the a behavioral economist enjoys too much undisciplined freedom of choice when picking a model to describe observed behavior (prospect theory, beta-delta discounting, etc.).
In my opinion, it is perfectly fair to question any behavioral economist about the circumstances when a particular type of bias should be more likely to occur. That is, he or she should be able to put falsifiable predictions on the table. But it is also fair to question the tendency to rationalize any observed behavior, which has been known to happen among some advocates of purely rational economic models.