Some of the city’s most distinguished academics, historians and authors have spoken out against the removal of the statue.