Originally built by the Hussey family, this fortification changed hands dramatically after the Desmond Rebellion when it was confiscated and granted to the Earl of Ormond. The Knight of Kerry later purchased the property, and it evolved into the Market House, serving as the official residence of Dingle's Sovereign whilst its vaults functioned as the town jail until 1815.

The castle's transformation from medieval stronghold to civic building reflects the turbulent history of this Kerry town. When John Hussey forfeited the building in 1585, the proposed town charter designated it to become both a jail and a common courthouse, cementing its importance in local administration. By 1841, little trace remained of the original structure, though mid-19th century observers noted tantalising architectural fragments; a pointed medieval doorway preserved in the ruins, and a low, dark entrance with a small grated window built into what were then modern walls.

Today, visitors to P.T. Fitzgerald's Supermarket might miss the sole surviving relic of this once-important castle. Embedded in the east-facing wall of an outbuilding at the rear of the premises, at second floor level, sits a carved stone bearing 16th century decoration. Its relief carving shows two intertwined ivy leaves with interlaced stems, a delicate reminder of the medieval craftsmanship that once adorned this seat of power. Archaeological investigations in the 1950s uncovered a lintelled passageway leading to a small chamber, possibly part of the original complex's back house, though these features have since vanished beneath modern development.