The house once stood on the flat valley floor, positioned strategically between the flood plain and the sloping valley sides, offering pleasant views to the east, south and west. Historical records from the Down Survey of 1655-6 show both a castle and ‘a stone house’ at this location, with the accompanying terrier noting the presence of ‘a stone house in repaire and cabbins’ at Lisdowny.

The house belonged to the Lords Mountgarret and was typically occupied by junior branches of the family. According to local historian Carrigan, writing in 1905, the last resident was a Miss Cheevers who died around 1809; shortly afterwards, the house was accidentally destroyed by fire. The walls were subsequently dismantled almost to ground level, the outbuildings removed, and the garden boundaries levelled. When inspected in 1987, the foundations were still visible, revealing a substantial structure measuring internally 15 metres east to west and 6 metres north to south, with walls nearly a metre thick built from stone and extremely hard grouting.

The house sat within a large sub-rectangular enclosure measuring 87 metres north to south and 72 metres east to west, bounded by a field bank on the west side and stone walls elsewhere. Foundations of an additional range, measuring 28 metres by 8 metres, extended northwards from near the southwest corner of the house. Both structures predate the stone-walled enclosure. Whilst the house appeared on the first edition Ordnance Survey map of 1839, it had vanished by the 1900 revision. Recent satellite imagery from 2018 suggests that even the enclosing walls and remaining building foundations have now been levelled, with no visible trace of the castle mentioned in the 17th-century records remaining at ground level.