I still haven't seen Season 4. Pete is such an interesting character because he's not really particularly likeable or sympathetic, but the show does a brilliant job of showing how the universe has shaped him into being what he is, so you do sympathize, a bit, to a certain point. (And then you stop.)

And I agree, the office lost so much life when Joan left. I was so happy at the last episode.