By the time cartographers got round to properly documenting it on the 1836 Ordnance Survey map, it appeared as a modest rectangular structure, roughly 10 metres by 5 metres, with a small projection on one side and an attached rectangular enclosure measuring about 25 by 20 metres.

Perched on a low ridge running north to south, the castle site underwent significant changes over the centuries. When archaeologists examined it in 1970, they found a raised, grass-covered oval platform measuring 41 metres north to south and 23 metres east to west, defined by low banks sitting atop natural scarps that varied in height from 2.3 metres on the northern side to 3.8 metres on the western edge. The actual castle structure, visible at the southeastern corner, had been reduced to a small rectangular foundation with interior dimensions of just 2 by 4.2 metres, its grass-covered banks still showing traces of stone facing. A small rectangular enclosure, measuring 8 by 6 metres, was attached to the western side.

By 1984, even these modest remains had largely vanished, leaving only a broad, grassy platform that appears to be a natural feature of the landscape. Today, visitors to Crossdrum Lower would be hard-pressed to identify any obvious signs of the castle that once stood here, though the raised platform, measuring 55 by 32 metres and reaching up to 2.5 metres in height, still marks the spot where this forgotten fortification once commanded the surrounding countryside.