The site appears on the Down Survey barony map from 1656-8, where it’s depicted as a one-storey gabled house with two chimneys at its centre. By the time the Ordnance Survey mapped the area in 1836, the structure had been reduced to a small square building measuring roughly 5 metres on each side.

The castle’s history can be traced back to at least 1640, when George Fleming owned 212 acres at Killary. According to the Civil Survey of 1654-6, Fleming’s property included “an old stone house and a church”, suggesting the building was already considered ancient by the mid-17th century. The church mentioned in the survey is likely the parish church, catalogued today as ME012-020, though curiously it doesn’t appear on the Down Survey map despite being noted in written records.

Today, visitors to the site will find little more than that subtle elevation in the pasture land, a quiet reminder of the castle that once stood here. The transformation from a substantial gabled house with twin chimneys to a small square structure, and finally to an almost imperceptible rise in the earth, tells the familiar story of Ireland’s vanishing medieval landscape, where centuries of abandonment, stone robbing, and agricultural use have gradually erased once-prominent buildings from the countryside.