The National Academies of Sciences, Engineering and Medicine
Policy and Global Affairs
Home About Committee Educational Modules Feedback Contact Us

Scientific Methods

The probabilistic nature of most empirical knowledge

Probability
Axioms
The concept of variables
Independence
The probability of independent and non-independent conjunctive events
Why P(A|B) is different from P(B|A)

The world as it actually is vs. the world as we discern it

Population vs. sample distributions
Descriptive statistics
The purpose of statistics: to infer the population distribution from a sample

Type I vs. Type II errors

Confidence internals/margins of error

Linear regression, simple and multivariate

Correlation coefficient

Correlation vs. causation

Validity

Measurement errors, reliability

Testing hypotheses

Null hypothesis statistical testing

Particular methods

Randomized controlled trials
Quasi-experimental designs
Observational studies
Econometrics
Qualitative methods

Replication

Declining effects

Bayes’ Theorem

Combining information
The idea of priors
A Bayesian approach to testing hypotheses

The difference between using statistics (1) to make generalizations about populations and (2) to judge the cause of a particular event or predict an outcome for a particular individual

The ideal of falsifiability and its limits

Different disciplinary approaches

Is there a fundamental difference between natural and social science methods?
Survey data
Sampling and public opinion polling
The growing role of administrative data (e.g., credit card use) as an alternative to sampling
The role of models (e.g., in climate, astrophysics)
Differences among approaches of various natural sciences (e.g., physics, astrophysics, chemistry, biology, clinical medicine)

Publication bias

Biases in perceiving, remembering, and analyzing information (judgment and decision making and social psychology)

Lay intuitions about probability and statistics
Confirmation bias
Anchoring, availability, representativeness, hindsight, etc.
Social influences

Scientific Culture and Communities

Nature of scientific consensus

The contingent nature of scientific standards and findings

Who sets standards?

Peer review and publication/circulation of data and results

Publication bias, replication

The tenure process

Paradigm-shifting visionaries vs. cranks

Role of learned societies

Role of NSF/NIH/other research agencies

Science Advisor/OSTP

Resolving scientific disputes

Scientific ethics:

conflicts of interest, misbehavior
self-regulation
external regulations

Science and Society

What constitutes a scientific consensus and what are the implications for policy and business decisions?

Decision-making under uncertainty

Lay vs. expert, global vs. domain specific views of risk

Precautionary principle

Scientists’ involvement in policy and litigation
Issues of responsibility, expertise, and credibility

Science in the courts

Admissibility
Daubert, etc.
State of forensic science

Examples of Subject Areas Where Scientific Understanding is Critical to Making Informed Decisions

DNA

Genomics

Breast, prostate cancer and other screening

Forensic science

Water Quality

Electromagnetic radiation

Emerging technologies

Intellectual Property

Response to global disease

Scientific issues in remediation of environmental harm

Consumer genetic screening

Gene editing

Fetal testing and genetic screening for newborns

Genetically modified foods and health and safety

Advertising and labeling of food, drugs, etc vis-à-vis health and safety claims

Climate change

Health care

End of life

Reproductive rights

Stem cell research

Nuclear weapons proliferation and nuclear weapons security

Food safety

Environmental standards

Internet

Surveillance of potential terrorism through mathematical data mining of “private”  information
Cybersecurity