He worked for traveling theatre orchestras

Sousa remained in Washington for a while after being released from the Marine Corps, directing and playing the violin. He traveled with a number of traveling theater orchestras before relocating to Philadelphia in 1876. He was employed by publishing businesses there as a composer, arranger, and proofreader. Sousa was enthralled by the operetta genre and went on tour with a group performing the musical Our Flirtation, for which he composed the march and the accompanying music. He received a telegram offering him the direction of the Marine Band in Washington while on tour in St. Louis. On October 1st, 1880, he accepted and reported for duty, becoming the band's 17th Leader.

Sousa's first experience leading a military band was the Marine Band, and he addressed musical things differently than most of his predecessors. In order to suit his demands, he substituted symphonic transcriptions for a large portion of the library's musical selections. He molded his musicians into the nation's best military band, and practices became incredibly rigid. Concerts by the Marine Band started to draw discerning crowds, and word of the band traveled quickly.

When Sousa composed "The Gladiator" in 1886, military band circles began to recognize him. He started getting more and more respect and attention as a composer after that. He composed "Semper Fidelis" in 1888. It is regarded as the "official" march of the Marine Corps and is dedicated to "the officers and men of the Marine Corps."