Yet that remnant is enough to read the logic of a building that grew around something far older, cannibalising a medieval tower house rather than replacing it. The new house was raised against the eastern and southern faces of the tower, absorbing it so completely that doorways were punched through to connect the two structures, the tower's upper windows were altered to suit the arrangement, and a brick fireplace and flue were inserted directly into the tower's exterior face at first-floor level. Tower houses, the fortified stone residences common across later medieval Ireland, were typically four storeys tall and built for defence; here one was quietly domesticated and made to serve as the backbone of a gentleman's residence.

The Shortall family held the castle until the Cromwellian Land Settlement of the 1650s and 1660s, the upheaval that redistributed vast tracts of Irish land to English settlers and soldiers. The St. Georges took possession after that and appear to have occupied and extended the earlier structure for roughly a century and a half before commissioning the present Kilrush House, completed around 1818. An estate map of 1750, still kept in Kilrush House, offers a precise snapshot of what the earlier arrangement looked like: a four-storey tower house standing alongside a two-storey house of three bays, with a steeply pitched roof rising to the level of the tower's third storey and two chimneys. Of that house, only the western wall now survives upstanding, and not to its full height. It has been neatly refaced at some point, probably when a small single-storey extension was added to the west. Brick voussoirs frame a flat-headed window in the southern face, and a two-course brick eaves runs along the top. The tower's original entrance, a round-headed doorway in the north wall, retains a wooden door thought to date from the late 16th or early 17th century. One further object connects the ruins to the later house: a Kilkenny marble fireplace, made from the dense black limestone quarried locally and long prized for interior fittings, which is now installed in the 19th-century Kilrush House and is said to have originated in this earlier structure. A medieval building lies around 110 metres to the east-northeast, and a medieval church roughly 145 metres to the north-northeast, so the site sits within a cluster of occupation reaching back well before the Shortalls or anyone who came after them.