The physical appearance of the Castle changed in the 18th century. The stair tower of the Castle was demolished around 1760 when it was remodelled as a more comfortable residence. A warehouse was built onto the castle around 1830 and was occupied by Joseph Doyle, a seed merchant and florist. The Ordnance Survey Memoirs of 1834-6 mention that the Castle was occupied as two dwelling houses and describe fragments of carved stones from the abbey buildings having been reused in surrounding buildings.
Having established a successful bakery in Castle Street sixty years previously, Arthur McCann Limited bought the site in 1894. It was described in the deeds as Castle, Orchard and Garden. Over the years many alterations have been made to the building to accommodate McCann's expanding business. In 1947, when new ovens were installed to the back of the Castle, many human bones were found. The alterations disguised the origins of the building and, for many years, the only clue to the site's significance was the stone carvings preserved in the bakery's walls.
A granite carving of a human head and heraldic animal was set into the outside walls, and later moved inside the bakery. A headless effigy of a knight in armour stood at the back of the bakery but is believed to have been re-buried in the grounds. A rough-hewn holy water font was removed from the grounds to the local Dominican Church.
15th century carving of a head and heraldic animal. Originally one of a pair, the carvings would have been a decorative feature situated on either side of a window or doorway in the Cistercian Abbey
The carving has been cleaned and conserved by The Northern Ireland Environment Agency.