Rationally Speaking:

The main problem, as I see it, is that while we can usually count on the process of peer criticism to check the research within a field for validity, I’m not sure we can count on it to check the validity of the field itself. I’m talking here about the methodology and the philosophical assumptions that underlie a field, the usually-unstated pillars on which all its findings rest.

Why? First of all, there is no incentive for a member of a field to challenge those foundational assumptions. It will be extremely difficult to get other people to agree with you, and even if they do, you’re still taking yourself down along with the rest of the field. Second of all, fields naturally select for people who accept their foundational assumptions. People who don’t accept those assumptions are likely not to have gone into that field in the first place, or to have left it already. …

So we almost seem to be stuck in a Catch-22: The only people who are qualified to evaluate the validity of a complex field are the ones who have studied that field in depth — in other words, experts. Yet they are also the people who have the strongest incentives not to reject the foundational assumptions of the field, and the ones who have self-selected for believing those assumptions. So the closer you are to a field, the more biased you are, which makes you a poor judge of it; the farther away you are, the less relevant knowledge you have, which makes you a poor judge of it. What to do?