While local tradition suggests the Earl of Desmond built the castle around 1400, the site likely hosted fortifications from the early 13th century when it served as an important Anglo-Norman manor. The complex consists of an impressive octagonal donjon, possibly dating to the 1200s, later enclosed within substantial bawn walls that may have been added in the 15th or 16th century. The octagonal tower, measuring nearly 6 metres in diameter internally, originally rose four storeys high and featured an unusual anti-clockwise spiral staircase; a rarity in Irish castle architecture.

By 1584, the Desmond Survey painted a picture of decay, describing the castle as "very ruinous" with neither room nor roof, though the surrounding manor lands remained remarkably fertile. The fortifications consisted of a rectangular bawn measuring approximately 46 metres by 55 metres, with walls reaching up to 7.6 metres in height and 1.5 metres thick, built from squared limestone blocks bound with high-quality mortar. The complex included at least five separate structures within or attached to the bawn walls, including polygonal towers at the corners and a pointed gateway entrance on the south side, complete with a bar-hole for defence.

The castle's strategic importance continued through the centuries of conflict; in 1593, Edward Fitton, Sheriff of Limerick, fled leaving Glenogra to rebel forces, and in 1600, the town belonging to Sir George Bourchier was laid waste by Piers Lacy from Kilquig. By the mid-17th century, Henry, Earl of Bath held the property, which then boasted not just the castle and bawn but also thirty houses and cabins, a mill, and manorial courts. Today, though much of the facing stone has been removed over the years and several towers have collapsed, the eastern walls of the bawn and portions of the octagonal tower still stand, offering visitors a glimpse into the layered history of this once-formidable Desmond stronghold.