It’s because the ver looks like vert which is the french word for green

Vermont is derived from Green Mountain

So vermilion looks like it could mean a green million, which sounds like a funny way to say a million dollars.