In 1247, Thomas de Bosco held both Ballyconnick and nearby Ballyanne in Bantry through knight's service from the Valence estate. By 1425, the same family, now anglicised to Bosher, still controlled both areas, though the estate's value had fallen by half; a decline that likely indicates Ballyanne had been abandoned by this time.

The Bosher family's fortunes continued through the centuries, though not without challenges. Between 1551 and 1559, Jasper Boshere of Ballyconnick took legal action against Cahir Mc Art Kavanagh in attempts to reclaim Ballyanne, but despite winning his case, he never managed to regain actual possession of the land. The family maintained their presence at Ballyconnick throughout the late 16th and early 17th centuries, with Nicholas Bosher appearing in records from 1571, 1574 and 1618. By 1640, Robert Busher had expanded the family holdings significantly, owning not just the castle and 240 acres at Ballyconnick, but an additional 460 acres spread across the parish in areas known as Croase, St. Tenants, Edwardstown and Philippintown.

Today, nothing remains of the castle that once stood on this gentle southeast facing slope. When antiquarian John O'Donovan visited around 1840, he found only a portion of wall that had been incorporated into a haggard wall, and even this fragment has since vanished. The site now forms part of a working farm complex, with no visible traces of the medieval structure that once represented the power and influence of the Bosher family in south Wexford.