Local tradition claims the O’Donovan clan first built a fortification here, though the historian John O’Donovan himself expressed doubts about his namesake family’s connection to the site. What seems more certain is that some form of pre-Norman stronghold existed here, possibly at the strategic river crossing point; the Annals of the Four Masters record that in 1151, Ruaidhri Ua Conchobhair burned ‘Cromadh’ during a cattle raid into Thomond.

The castle as we know it today began taking shape in the late 12th or early 13th century under Gerald FitzMaurice of the powerful FitzGerald family, who likely died around 1203. The first documented reference to Croom Castle appears in 1215, when Maurice FitzGerald paid King John 60 marks to inherit his father’s lands, including the castles of Croom and Dungarvan. The FitzGeralds, who would become Earls of Kildare, constructed their shell keep castle during a period of intense Anglo-Norman castle building across Limerick between 1200 and 1220, establishing strongholds at Kilmallock, Caherconlish, Ardpatrick and elsewhere. The castle remained in FitzGerald hands for centuries, though not without interruption; during the 1534 rebellion of Silken Thomas, the Earl of Desmond briefly seized Croom and garrisoned it, whilst his son James later secured a lease from Henry VIII for both Croom and Adare manors.

By 1600, Sir George Carew described finding thatched houses within the castle’s bawn walls, which remained ‘good and firm’ despite the buildings being accidentally burnt during military action. The Civil Survey of 1654;56 recorded a castle with its bawn, an orchard, a derelict mill, two eel weirs, and a broken bridge across the Maigue, still nominally owned by the Earl of Kildare. After passing through various hands, including Thady Quin of Adare in 1683 and the Duke of Richmond in 1707, the Croker family of Ballyneguard acquired the property in 1711. Left to decay after 1691, the medieval fortress was eventually incorporated into a 19th century castellated mansion by John Croker, whose Croom Castle House now encompasses much of the original footprint.