Originally depicted in Thomas Raven’s 1634 survey, the castle sat within a rectangular bawn featuring circular corner towers at its southern and western angles, with a fortified gateway between them. The defensive walls included smaller postern gates, whilst stables and outhouses were built directly into the bawn’s longer walls. Though the castle was burnt in 1688, it continued serving as the administrative base for land agents of the Weymouth and later Bath estates, maintaining its essential structure well into the 18th century according to a 1736 drawing.

The castle’s fortunes shifted dramatically in 1851 when William S. Trent became land agent and undertook substantial rebuilding work. He likely constructed the block facing Castle Street, cleverly incorporating the original corner towers and gatehouse into his design. When the Bath estate sold to its tenants in 1888, Rev. Peter Birmingham, the local Catholic parish priest, purchased the castle and surrounding lands for the Sisters of Saint Louis. The sisters transformed the site into a boarding school for girls in 1890, which continues to operate today as a day school, though the religious order is no longer involved.

The original medieval structures remain hidden in plain sight within the Victorian additions. The gatehouse appears to be incorporated into the castellated central block of what served as the convent, whilst the corner towers are likely entombed within the large polygonal bay windows at the northwestern and southeastern ends of the building. The original bawn would have occupied roughly the area where the school now stands, measuring approximately 60 metres northeast to southwest and 35 to 40 metres northwest to southeast, between the convent building and Distillery Lane. Archaeological testing in 2000 and 2004 in the surrounding area yielded no related material, leaving the exact extent of the original complex somewhat mysterious.