He married his wife before she had been legally divorced from her first husband

In the 1780s, after relocating to Nashville, Tennessee, Andrew Jackson developed feelings for the unhappy married Rachel Donelson Robards. Robards married Jackson after divorcing her spouse and assuming that she legally divorced her first husband. Her first husband, though, had accused her of adultery while the divorce was still pending. Jackson formally wed Robards again in 1794.

However, during the bitter 1828 presidential campaign, Jackson's political rivals propagated rumors about his wife's alleged adultery. They remarried after realizing their mistake, but the damage had already been done. Jackson's moment of victory was quickly eclipsed by personal sadness when, on December 22, 1828, his wife passed away at the Hermitage.

She had shied away from the spotlight because she was retired and religious, and the scandalous attacks had devastated her. On December 22, 1828, she passed away. The loss of her crushed Andrew. Jackson always attributed her passing to his political rivals, despite the fact that her illnesses had started as early as 1825. Andrew Jackson's gravestone bears the following description: "A being so gentle and yet so virtuous, slander might wound, but could not dishonor.” She had expressed her reluctance to accept the position as hostess of the President's House, stating that she would "rather be a doorkeeper in the House of God than live in that palace."