First mentioned in a deed from 1334, this castle was built by Sir John Cruise before passing into the hands of the Fitzwilliam family in the early 1400s. When Sir Thomas Fitzwilliam made it the family seat around 1550, the castle became their ancestral country home for over 150 years, surviving damage during the 1641 rebellion and subsequent restoration before finally being abandoned in 1710 when the 5th Viscount Fitzwilliam relocated to a new Merrion House in Mount Merrion.

Historical records paint a vivid picture of the castle’s evolution and appearance. A manuscript map by John Barker shows it as two adjacent crenelated towers; one three storeys high, the other two. The Down Survey of 1655-6 describes it as a ‘faire stone castle and a grove of trees’, whilst a 1766 drawing by Beranger reveals a roofless hall-like structure with stepped battlements, a wall-walk, and a roofed corner tower at the north end. The three-storey facade facing the sea featured both flat-headed and round-headed windows, some with hood mouldings, whilst the landward side was punctuated by closely spaced large openings. By 1780, Austin Cooper noted that the ground floor had been converted to a cow-house with parts of the outlying buildings serving as stables, though Duncan’s map of 1821 confirms the castle was still standing.

Today, fragments of this once-grand estate remain scattered across the site. An armorial plaque bearing the Fitzwilliam Coat of Arms has been incorporated into the south gable of Fitzwilliam Hall, a newer building constructed where an old coach house once stood. A carved stone head and an 18th or 19th century house also survive within the grounds. Perhaps most intriguingly, Barker’s map depicts an unusual D-shaped enclosure bounded by a curving earthwork with one edge along the shore; by the time of the first Ordnance Survey maps, this location featured a serpentine fish pond, adding another layer to the site’s rich history.