The Mac Ui Brien Aradh branch of the Clann Brian Ruadh established themselves here after crossing the Shannon from Clare following their victory at the Battle of Dysert O’Dea in 1318. The strategic location gave them control over both land routes and the valuable eel weirs along the Shannon, with four weirs specifically mentioned in historical records as belonging to the castle lands between Ballina and Killaloe.

The castle gained formal recognition in 1606 when King James I granted Murtagh O’Brien extensive lands across Limerick, Tipperary, Clare and Galway as part of the surrender and regrant scheme, including the manor, town and lands of Castleton in the cantred of Arra. This grant came with significant privileges including the right to hold a Thursday market at Castleton, a fair at Pallaice on St Luke’s day, and both a monthly court baron at Castleton and a half-yearly court leet at Pallaice. By the 1650s, the Civil Survey described a thriving settlement with the castle accompanied by a gatehouse, slate house, orchard, garden, watermill, eight thatched tenements and the parish church, all under the ownership of Donogh O’Brien of Ballina Castle, though the property was mortgaged to Richard, Earl of Corke, in 1640.

The surviving structure is a substantial three-storey rectangular block measuring 13.35 metres east to west and 6.5 metres north to south, with walls 1.75 metres thick. Archaeological evidence points to seventeenth-century construction, particularly visible in the two-centred arch doorways with punch-dressed chamfered jambs showing horizontal tooling. The ground floor retains part of its original barrel vault, complete with impressions of wicker centring on its underside, whilst the upper floors once featured wooden floors supported on stone corbels. Two fireplaces survive in the north wall; one partially destroyed at first-floor level and another well-preserved example on the second floor, served by a tall rectangular chimney stack. Though modified by a nineteenth-century house addition at the southeast angle and external stone stairs to a first-floor entrance, the castle remains an impressive testament to the O’Briens’ centuries of influence in the region.