Gravitational wave observations of compact binary mergers are already providing stringent tests of general relativity and constraints on modified gravity. Ground-based interferometric detectors will soon reach design sensitivity and they will be followed by third-generation upgrades, possibly operating in conjunction with space-based detectors. How will these improvements affect our ability to investigate fundamental physics with gravitational waves? The answer depends on the timeline for the sensitivity upgrades of the instruments, but also on astrophysical compact binary population uncertainties, which determine the number and signal-to-noise ratio of the observed sources. We consider several scenarios for the proposed timeline of detector upgrades and various astrophysical population models. Using a stacked Fisher matrix analysis of binary black hole merger observations, we thoroughly investigate future theory-agnostic bounds on modifications of general relativity, as well as bounds on specific theories. For theory-agnostic bounds, we find that ground-based observations of stellar-mass black holes and LISA observations of massive black holes can each lead to improvements of 2-4 orders of magnitude with respect to present gravitational wave constraints, while multiband observations can yield improvements of 1-6 orders of magnitude. We also clarify how the relation between theory-agnostic and theory-specific bounds depends on the source properties.