In perceptually degraded situations, state estimation drift is a major source of failure for autonomous operations of Micro Aerial Vehicles (MAV). This paper serves as a guide for understanding and characterizing estimation drift during dead-reckoning navigation with systems using onboard MEMS inertial measurement units. The analysis is conducted with the commercial-off-the-shelf Pixhawk flight controller, running the commonly-used PX4 autopilot. The performance of the Extended Kalman Filter (EKF2) and Local Position Estimator (LPE+Q) were characterized through a two-step experiment. First, the onboard IMU is manually excited while the estimators receive state updates from a motion capture system. In a second step, a dead-reckoning scenario is created removing the state updates in the estimators. Heat control was also added to analyze temperature effects on the IMU biases and hence estimation drift. Our analysis shows how the estimation drift depends on the quality of the IMU, excitation of the IMU, the estimation algorithm, and temperature control.