Monroe’s later years were difficult for him and his family

When he was younger, James Monroe came to the realization that, unlike his father and grandpa, he wasn't a decent farmer. He chose to enter national politics in order to seek a secure political career, thus he sold off the majority of his land. Finally retiring in 1825, Monroe relocated to Virginia. However, he was unable to remain silent, so in 1829 he assisted in revising the Virginia Constitution.

He served on the university's Board of Visitors under both Thomas Jefferson and James Madison, the second rector. Up until his wife's passing, he and Elizabeth resided at Oak Hill in Aldie, Virginia. In 1830, his wife passed away, profoundly affecting Monroe.

By the end of the 1820s, Monroe's health had started to deteriorate gradually. James Monroe became the third president to pass away on Independence Day when he succumbed to heart trouble and tuberculosis on July 4, 1831, at the age of 73. His passing occurred five years after the passing of John Adams and Thomas Jefferson and 55 years after the proclamation of the United States Declaration of Independence. His body was reburied at the President's Circle in Hollywood Cemetery in Richmond, Virginia, 27 years later, in 1858. Among the nation's National Historic Landmarks is the James Monroe Tomb.