We present mathematical and conceptual foundations for the task of robust amplitude estimation using engineered likelihood functions (ELFs), a framework introduced in Wang et al. [PRX Quantum 2, 010346 (2021)] that uses Bayesian inference to enhance the rate of information gain in quantum sampling. These ELFs, which are obtained by choosing tunable parameters in a parametrized quantum circuit to minimize the expected posterior variance of an estimated parameter, play an important role in estimating the expectation values of quantum observables. We give a thorough characterization and analysis of likelihood functions arising from certain classes of quantum circuits and combine this with the tools of Bayesian inference to give a procedure for picking optimal ELF tunable parameters. Finally, we present numerical results to demonstrate the performance of ELFs.