It is not known exactly when the construction of the castle started but it is believed that it took place around the late 1290s, by King Haakon V, replacing Tønsberg as one of the two most important Norwegian castles of the period (the other being Båhus). It was constructed in response to the Norwegian nobleman, Earl Alv Erlingsson of Sarpsborg’s earlier attack on Oslo that occurred in 1287. In the aftermath of the attack, it became clear that the city’s existing defences weren’t effective and therefore, a stronger defensive centre was needed.
The castle is mentioned in written sources for the first time in 1300 in a letter from King Haakon to a church in Oslo. However, the letter does not mention how far the construction of the castle had progressed by then.
The fortress was first used in battle in 1308, when it was besieged by the Swedish duke Eric of Södermanland, whose brother won the Swedish throne in 1309. The siege was eventually broken by a local Norwegian army in a battle. (This battle forms a major part of the plot of Sigrid Undset's historical novel In the Wilderness, the third volume of her tetralogy The Master of Hestviken.)
In 1449-1450 the castle was besieged again, this time by the Swedish king Karl Knutsson Bonde, but he had to lift the siege after a while. The castle was not besieged again until 1502 when Scottish soldiers in the service of the Danish king besieged the castle in order to regain it from the hands of the Norwegian nobleman Knut Alysson.
Akershus was besieged yet again in 1523, this time by Swedish soldiers but Oslo’s inhabitants burned down their houses in an attempt to drive them out and the Swedes retreated after a short period. King Christian II besieged the castle from 1531 to 1532 but the siege was lifted by forces from Denmark and Lübeck. After this siege the castle was improved and strengthened.
In 1567, during the Northern Seven Years' War, the castle was besieged once more by Swedish forces, but the Danish king's lord lieutenant, Christen Munk, responded by burning down the city in order to deprive the attackers themselves of the means of receiving supplies, and eventually the Swedes retreated.