The structure likely incorporates elements of a fortress first built in 1467, when John Tiptoft, Earl of Worcester and Lord Deputy for Edward IV, leased the land specifically to construct a defensive stronghold. The original purpose was refreshingly practical: to prevent pirates from using the island as a base for plundering ships traversing the Irish Sea, a genuine problem that plagued maritime trade routes during the medieval period.

The castle's distinctive design features four projecting angle towers, strategically positioned to defend all the walls; a defensive architecture that speaks to 16th or 17th century military engineering. These towers, complete with triangular musket holes for flanking fire, transform what might have been a simple fortification into a formidable defensive structure. The two storey building centres around a rectangular main chamber measuring roughly 9 by 4.5 metres, entered through an eastern doorway. Interestingly, the western wall's greater thickness at 1.3 metres suggests it may be part of an even earlier structure, hinting at layers of history yet to be fully uncovered.

In the early 20th century, the renowned architect Edwin Lutyens incorporated this historic castle into his grand redesign of Lambay, seamlessly blending centuries of defensive architecture with his own vision. Each angle tower, squared internally but splaying externally, contains its own fireplace and is topped with a distinctive crow foot gable supporting a chimney. While later modifications have altered some original features, such as the insertion of a fireplace blocking access to the southwest tower and the remodelling of the northeast corner into a drawing room, the castle remains a remarkable testament to Ireland's complex history of coastal defence and architectural evolution.