Blaenllyfni Castle was the centre of the Fitz Herbert Barony of Talgarth which was only created as a third part of Brecon lordship in 1208. Consequently the fortress was probably constructed in the years 1208 to 1215, after which it fell briefly into the hands of the Braose family.
It was returned to the Fitz Herberts in 1217/8 and was sacked by Prince Llywelyn ab Iorwerth and Richard Marshall in the October of 1233. Rebuilt soon afterwards it was apparently taken by Prince Llywelyn ap Gruffydd late in 1262. It was retaken by September 1273 when Reginald Fitz Peter was rebuked for his castle taking activities in Brycheiniog. The castle was seized by the Crown after the abortive uprisings of 1321-2 and given to the Dispensers until their overthrow late in 1326.
The castle by this time was nearly ruinous and an inquisition by jury of 23 January 1337 held at the castle found numerous defects which suggests that the castle had never recovered from the attentions of Llywelyn ap Gruffydd, even if the archaeological evidence does suggest that the castle defences were improved at this time. The castle is on private grounds. There are remains of a keep and several towers.
