I felt for that trap at least twice - one making an engine in OpenGL and another in Direct-X 😁
Its a great way to learn how things works under the hood!
I only tend to avoid building a custom engine with 3D, too many complexities that can go wrong.
For 2D, its another story. Much simpler to get something going quick :)