His chief rivalry was with his father-in-law!

Earl Godwin of Wessex had become one of the most powerful Earls in England during the reign of Edward’s predecessor, the Danish king Cnut the Great. His daughter Edith of Wessex married Edward the Confessor in 1045. When Cnut died in 1035, his lands were divided among three contenders, Alfred Aetherling being one of them (and also the brother of Edward).

With Earl Godwin's backing, he usurped the throne from Harold Harefoot, who had assumed it. Edward believed Godwin was responsible for Alfred's blindness and death since it was said that Godwin either kidnapped Alfred or tricked him into thinking he was an ally before giving him to Harold. Despite the absurd fact that Edward the Confessor married Godwin's daughter, this resulted in a rivalry that was never forgotten.