Sculpted by Edward Smyth, it is, according to Wheeler and Craig, the only statue on any Protestant church in Dublin. It has not had an easy existence: the notes record that it suffered considerably from the weather and from having been used for target practice by a noted duellist. The church it guards has since been converted into a licensed food hall, and most people passing through it on a given afternoon would have little reason to suspect the layered and repeatedly catastrophic history underneath their feet.

The parish itself was probably founded in the eleventh century, its original medieval church standing just outside the old city walls on what is now Dame Street, on the site later occupied by a bank. After the Reformation the building was repurposed as a mint and then a vice-regal stable, and the parish was not formally re-created until an Act of Parliament in 1665. The replacement church, built between 1670 and 1674 on a former bowling green donated by Henry Jones, Bishop of Meath, was elliptical in plan, designed by William Dodson. It stood near the Thingmote, a Viking assembly mound of the kind used by Norse settlers for public gatherings, which was not levelled until 1685. Dodson proved a dishonest contractor, and the building was decaying by 1750. Parliament repeatedly bailed the parish out: grants came in 1745, 1796, 1799, and again in 1805, when £6,000 was approved for completing the rebuilding and erecting a steeple. Francis Johnston took over the project in 1800 after John Hartwell resigned, and the interior, finished in an Egyptian style and considered too well-lit for comfort, was fitted with oiled-silk blinds printed with scriptural scenes. A large carved wooden candelabra inherited from the House of Commons after the Act of Union was prudently moved to Trinity College's Examination Hall in 1852. On 8 January 1860, the church burned to the ground in an accidental fire. The surviving tower was demolished immediately, and the present building, designed by Lanyon, Lynn and Lanyon of Belfast following an architectural competition, was opened in 1866.

The current structure, a cruciform Gothic Revival building with a broach spire, a form in which the spire rises directly from the tower without a parapet, sits a few yards south of the old site, and shares nothing architecturally with what it replaced. Esther Vanhomrigh, known to readers of Swift as Vanessa, is buried somewhere in the graveyard, though her grave is unmarked. The building also partially overlies the site of the medieval Nunnery of St Mary de Hogges, noted on the Ordnance Survey maps. The church is accessible from the junction of St Andrew Street and Suffolk Street in central Dublin, and the Smyth statue beside the west door repays a close look, battered as it is.