Ah, no one told you… Central Park sits where an affluent black community once existed (Seneca village), but was seized using eminent domain and razed.
Seneca Village was 5 blocks
The 5 blocks seem like a drop in the bucket, they probably could have just carved it out and let it be right? Oh nevermind, the rampant racism, class war, and general greed got in the way.