The castle's history stretches back to at least 1616, when Robert Rawcettor of Bargy and Walter Bryan of Scarcarrig leased the surrounding land from Rosegarland. The site appears on the Down Survey map from 1655-6, preserved in the National Library of Ireland, marking its significance in the landscape during the Cromwellian period.

By the time of the Civil Survey in 1654-6, the castle was recorded as being in good repair and under the ownership of James Bryan, son of Walter, who had inherited 200 acres at Longgraigue. The survey provides a valuable snapshot of the property during a turbulent period in Irish history, when detailed records were being compiled to document land ownership following the Confederate Wars.

Today, visitors to Longgraigue House can still observe traces of the medieval castle in its structure. The northern wall features a distinctive base-batter, whilst the southeast angle preserves both base-batter and granite quoins; architectural elements that speak to the building's defensive origins. These surviving features have been carefully documented by the National Inventory of Architectural Heritage, offering a tangible connection to the families who once controlled this corner of Wexford over 400 years ago.