The site’s strategic position made it an ideal location for the early 14th-century castle of the D’Erley family, recorded in medieval documents as ‘Curia de Erleyestoun’ and ‘Castrum Erleye’. The D’Erleys’ fortunes took a dramatic turn in 1367 when John D’Erley joined the Black Prince’s army in Spain, only to be captured; the crippling ransom demanded for his release forced the family to sell their manor, passing both the property and the title of Baron of Erley to the Sweetman family by the late 1300s.

The Sweetmans held Castle Eve for nearly three centuries, transforming and expanding the medieval fortress into a substantial fortified house. The complex features a defensive bawn with an external moat, a three-storey rectangular house in the southwest corner, and a substantial mural tower at the northwest angle. A carved stone lintel, now housed in Rothe House in Kilkenny, bears the initials of William Sweetman and his wife Joan Walsh alongside the date 1628, marking one of their renovation projects. The family’s tenure came to a violent interruption during the Irish Confederate Wars when Cromwell’s troops allegedly battered the castle in the 1650s, killing all but one defender and forcing William Sweetman into transplantation to Connaught, though he appears to have returned after the Restoration.

Following various changes of ownership through the Baker and Conway families, the fortified house saw partial conversion into a farmhouse in the 19th century, with its northern portion remodelled for agricultural use whilst the remainder fell into ruin. The Down Survey map of 1655-6 depicts it as a substantial house with pitched roof and central chimney, testament to its former importance. Today, only fragments remain of this once-impressive stronghold; demolition work in the 1990s reduced much of the western house to stumps, though visitors can still trace the outline of the bawn, examine the surviving architectural details like the fine limestone fireplace, and imagine the centuries of county politics, family fortunes, and military conflicts that played out within these weathered walls.