Whilst a castle is clearly marked in Ballyanne parish on a Down Survey map from 1655-6, no physical traces of this structure remain visible today. The land itself has a documented history; in 1640, 110 acres at Rathgaroge were owned by Nicholas Dormer, described as an Irish papist from nearby Stokestown. However, historical records from that period make no mention of any castle or fortification at this specific location, leaving historians to wonder about the accuracy of the later mapping.

The mystery deepened with the 1839 Ordnance Survey six-inch map, which shows a faintly marked enclosure labelled in gothic lettering as the ‘Site of Fort Garret and Castle’. This tantalising reference suggests that by the 19th century, local knowledge or tradition still preserved memory of some sort of defensive structure, even if the physical remains had already vanished. The site sits on a gentle west-facing slope, now used as pasture land, where cattle graze unaware of the possible medieval drama that once unfolded beneath their hooves.

Today, even the most keen-eyed visitor would struggle to spot any evidence of the castle’s existence. Local residents have no knowledge of the fortification, and ground-level examination reveals nothing more than ordinary farmland. This absence of both physical and folk memory makes Rathgaroge particularly fascinating for those interested in Ireland’s lost heritage; places where documentary evidence points to structures that have been so thoroughly erased by time that they exist only in old maps and land records, ghost buildings in a landscape that has moved on without them.