A highly prominent fountain centrepiece for the waterfront public plaza at Lonsdale Quay will be permanently removed.