You are correct. The consensus is that it is not vegan because of the animal labour.
Veganism at its core is about abstaining from using animal products AND avoiding making animals a commodity.
People may have different levels of commitment though. So not everyone will be as strict about it as the definition specifies.