The manor of Rathmore changed hands through strategic marriages over the centuries, beginning with the de Verdon family in the late 14th century before passing to Sir John Cruise by 1406. When Cruise’s daughter Mary married Sir Thomas Plunkett, third son of Christopher Plunkett of Rathregan, the property entered the Plunkett family’s possession, where it remained until the 17th century when it transferred to the Blighs. Historical surveys from the 1650s paint a picture of the estate’s former grandeur; Robert Plunkett held the entire parish in 1640, and the Civil Survey records two roofed castles, a quarry, and several cabins on the premises, though the Down Survey only depicts a single structure.

The tower house itself, likely built by the Plunketts in the 15th century, showcases the defensive architecture typical of its era. This rectangular stronghold measures 11.45 metres northeast to southwest and 9.4 metres northwest to southeast, rising four storeys from its position built directly into the north side of the motte. The interior reveals clever medieval engineering with two barrel vaults running northwest to southeast over the first floor. Originally, access between the ground floor chambers would have been through wooden trapdoors, evidenced by surviving corbels, whilst embrasures in the southwest wall have since been broken out. The original entrance sits at the southern end of a projection from the southeast wall, accessed from the top of the motte itself.

Navigation through the castle required intimate knowledge of its layout, with a complex system of stairs connecting the floors. From the first floor, a newel staircase in the eastern angle provided access to upper levels from the northeast chamber, whilst a mural staircase in the southeast wall connected the southwest chamber to the same newel stairs. The entrance lobby featured both a mural staircase rising southward through the southeast wall and access to the newel stairs, though sadly the latter collapsed around 2014. A pointed doorway from the lobby led to the second floor hall, measuring approximately 7.9 by 5.55 metres internally, which served as the castle’s main living space with large windows in the northeast and northwest walls and a garderobe tucked into the western end of the northwest wall; a reminder that even medieval nobility required certain conveniences.