Current site is due to be demolished to make way for housing in new disruption to city’s long trading tradition