I was also a former retail worker, but I guess my experience differed slightly from yours.
My primary responsibility was not getting the carts. I had a lot of other things that I was responsible for throughout my working day that had to get done. But when we needed to get the carts, I would often be the one asked to go.
When people left the carts all over the place like animals, it made my job take longer. And that meant I had more to catch up on back in the store once I finished. Not to mention, being out in the hot sun, or the rain, or the snow was not my definition of a fun time, so I typically wanted to be done with it as quickly as possible.
People would just toss their carts into empty spaces, even making eye contact with me as they proudly sauntered back to their soccer mom SUVs as if they believed they weregiving me the gift of job security. Sometimes they also confused “shopping cart” for “trash can”. I hated those people. I honestly wished death on some of them. I still can’t stand people like that today and I make every effort to not be one of them now.