I was sad to discover that you get no resources back when deleting a structure built in an outpost.
But you do get everything (it seems!) back if you remove the entire outpost from the Beacon menu. Into your ship's hold.
I created a small outpost then removed it to check:
Hold mass is 1:1 before/after. Items in containers (solid storage, a storage box, a display case) all were returned. Items on the floor in the habitat fell to the surface of the planet.