Sieges Were Not Very Easy
One of the most well-known depictions of castle warfare shows an army outside the walls constructing trebuchets or catapults to launch rocks at the walls and destroy them. Any castle seemed to be doomed by siege weapons and artillery. Yet in reality, it wasn't like that.
Although artillery weapons were notoriously difficult to aim accurately, they were best deployed to target certain areas of the defenses. Doors were obviously the most vulnerable points of entry. But, walls were not the best because their thickness may vary from six to more than twenty feet.
This was the reason military sieges might linger for months. In a movie, a catapult might easily destroy a castle, but in real life, they may have been beneficial for striking structures inside the walls of a castle but not for getting inside the structure itself. A siege would frequently succeed only if the defenders were starved to death and allowed to retreat. Yet, the attackers also ran the same risk because they were in a foreign country and may become starved and worn out.
Siege weapons were difficult to work with and were typically constructed on location using materials found nearby. Dragging them through the countryside was too challenging, and it might also be tough to locate ammunition. Due of the time and expense involved, attacking armies would only resort to siege warfare as a last resort.