The first known mention of the castrum (fortified village built around a castle or fortified house) dates back to 1145. At the time, it belonged to the de Dun and de Péreille families.

In 1207, a certain Hugues de Laroque was one of the 35 co-lords of Mirepoix. A few years later, during the crusade against the Cathars, several inhabitants of the castrum were spotted at Montségur. "I saw Pierre de Laroque, knight of Laroque d'Olmes, come to Montségur and he went to see the Perfects...". Deposition of Gaillard del Congost du Razès before the Inquisition, 1244. After the defeat of the Occitan lords, their lands, like all those around them, were handed over to Gui I de Lévis, Simon de Montfort's right-hand man. His descendants held it until the French Revolution. Still mentioned in 1372, the castle was no longer mentioned at the beginning of the 16th century.