The castle remains stand just a metre or two above the valley floor, with a small stream flowing roughly twenty metres to the north. Today, visitors will find little more than fragments of what was once a substantial fortification; the castle survives primarily as a rectangular hollow measuring approximately 16 metres north to south and 15 metres east to west, with only portions of the north wall still standing.

The castle's history is firmly rooted in the complex web of Irish land ownership during the early modern period. According to the historian Carrigan, writing in 1905, the castle was either founded by or belonged to the Cody family. An Inquisition from 1609 reveals that Sir Richard Shee held Castletown from Richard Archdeacon, also known as Cody, as part of the manor of Galmoy. The property passed through the Shee family line from Sir Richard to his son Lucas, and then to Edmund Shee, who ultimately forfeited the townland during the Cromwellian conquest, a fate shared by many Irish Catholic landowners of the period.

Historical accounts provide intriguing details about the castle's original scale. The Ordnance Survey Letters of 1839 recorded that only the north wall remained standing at that time, noting its impressive thickness of seven feet, or just over two metres. Carrigan's later description suggests this wall once rose to a height of 30 to 40 feet, roughly 9 to 12 metres, and measured an extraordinary eleven feet thick at ground level. The castle stood within a rectangular bawn, a defensive courtyard wall typical of Irish tower houses, though this too has largely vanished into the Kilkenny landscape.