The castle appears on the 1654-7 Down Survey map as a tower house within a bawn wall, marked as ‘Black Castle’, whilst later maps from 1837 label it as ‘Desmond Castle’. Its location on high ground near the Camoge River made it a strategically important site, and it may have been the same castle granted to John de Grey as early as 1199, suggesting it had been a significant fortification for centuries.

Historical records paint a picture of the castle’s gradual decline and changing ownership. By 1583, the structure was described as completely ruined except for its stone walls and a great enclosure called the ‘balne’ or bawn, along with a water mill on the nearby rivulet. The property changed hands multiple times throughout the 17th century; it was confirmed to the children of Captain W. Apsley in 1589, held by F. Fitton in 1627, and by 1655 was in the possession of Thady Grady before being sold to Richard Coote. The castle earned its nickname ‘Black Castle’ to distinguish it from the nearby White Castle to the east, both of which appear on period maps of the area.

The site held considerable importance to the local community, with the ‘Towne Green’ shown on the Down Survey map located in the field immediately west of the church ruins. By the 18th century, the castle had become part of James FitzGerald’s estate before being purchased by Alderman B. Burton in 1703. Writing in the early 20th century, Westropp noted that the castle, variously called ‘Desmonds’, ‘Black Castle’, or ‘East Court’, had been completely levelled. FitzGerald had earlier described it as the noble ruins of a castle on the river, erected by the Earls of Desmond, but even these romantic ruins have since vanished, leaving only historical records to tell the story of this once formidable stronghold.