@kit_ty_kate I tend to just forbid the usage of the Download directory and temporary stuff goes in ~/tmp instead.
Now about half of what I used to get is nicely sorted but ~/tmp is quite a mess.