In Sweden, yes tap water is safe without boiling. Might want to filter for taste reasons.

I would drink the water in Iceland without boiling it, probably without a second thought. In fact, I wouldn’t be surprised if it tastes amazing.