(1) Background: Low back disorders are a leading cause of missed work and physical disability in manual material handling due to repetitive lumbar loading and overexertion. Ergonomic assessments are often performed to understand and mitigate the risk of musculoskeletal overexertion injuries. Wearable sensor solutions for monitoring low back loading have the potential to improve the quality, quantity, and efficiency of ergonomic assessments and to expand opportunities for the personalized, continuous monitoring of overexertion injury risk. However, existing wearable solutions using a single inertial measurement unit (IMU) are limited in how accurately they can estimate back loading when objects of varying mass are handled, and alternative solutions in the scientific literature require so many distributed sensors that they are impractical for widespread workplace implementation. We therefore explored new ways to accurately monitor low back loading using a small number of wearable sensors. (2) Methods: We synchronously collected data from laboratory instrumentation and wearable sensors to analyze 10 individuals each performing about 400 different material handling tasks. We explored dozens of candidate solutions that used IMUs on various body locations and/or pressure insoles. (3) Results: We found that the two key sensors for accurately monitoring low back loading are a trunk IMU and pressure insoles. Using signals from these two sensors together with a Gradient Boosted Decision Tree algorithm has the potential to provide a practical (relatively few sensors), accurate (up to r² = 0.89), and automated way (using wearables) to monitor time series lumbar moments across a broad range of material handling tasks. The trunk IMU could be replaced by thigh IMUs, or a pelvis IMU, without sacrificing much accuracy, but there was no practical substitute for the pressure insoles. The key to realizing accurate lumbar load estimates with this approach in the real world will be optimizing force estimates from pressure insoles. (4) Conclusions: Here, we present a promising wearable solution for the practical, automated, and accurate monitoring of low back loading during manual material handling.