I don't care as much about non-square checkboxes (not that interested in talking about bad design), but if there are related design topics to think about, it is these:
- How to signify a radio button that can be unselected?
- How to signify checkboxes with a max count of possible to select elements? (“select 2 out of 5”)
- How to signify checkboxes with a min count of items (“select at least 1 element”)