Averell was later criticized for missing a golden opportunity to destroy Fitzhugh Lee’s brigade

The Confederate forces were in shambles by the conclusion of the day. On the Union left, Duffié disobeyed Averell's orders to hold his position and ordered a charge.

Lee's soldiers were worn out and dejected after numerous unsuccessful charges, and Averell maintained a definite advantage in manpower and momentum. Lee had to retreat his soldiers through the trees to a clearing close behind him due to the surprise attack. Averell did not, however, give the command for the decisive charge that might have eventually overcome Lee. He decided to retreat back over the Rappahannock after hearing what he believed to be the noises of southern troops being hurried to the battleground. After the conflict, Thomas Rosser said that Averell "mistook the frantic thumping of his frightened heart for drums." Even though he had fallen short of a complete triumph, Averell's actions that day had a significant positive impact on Union morale and raised alarm in the ranks of the South. He also left a sack of coffee and a note for his old friend south of the river: “Here’s your coffee. Here’s your visit. How did you like it? How’s that horse?”