He and Napoleon Bonaparte had mixed relationships
On the one hand, Lafayette and his family's escape from jail were thanks in part to Napoleon and his administration, the Directorate. On the other hand, they would not let Lafayette travel back to France until he had sworn fealty to the new regime. Lafayette was compelled to remain in exile in Germany until 1799 because he refused. He came back to France that year under a fictitious name. Napoleon was angered by this but was unable to punish him without upsetting the public. Lafayette, who kept a low profile throughout the Napoleonic Wars, was instead marginalized by Napoleon.
Surprisingly, Lafayette dissented from the new limitations on French democracy following Napoleon's first defeat in 1814 and did not support Louis XVIII. He afterward took retirement from public life in part as a form of protest against Napoleon Bonaparte's autocratic rule, and he went on to become a gentleman farmer. Lafayette rebuffed Jefferson's invitation to appoint him as governor of the recently acquired Louisiana Territory in 1803 when Jefferson was still the president of the United States. He was a member of the Chamber of Deputies throughout the majority of King Louis XVIII's reign (1814–24). Lafayette, however, rose to prominence during Napoleon's brief ascent to power and subsequent collapse in 1815, calling for his resignation and exile. Lafayette attempted to arrange for Napoleon to be granted refuge in the US, but his plans fell through due to British pressure.