The only thing you really need to worry about are the parts that the beer touches. So for caps, you can certainly soak them, but you don't really need to. (You don't soak the whole bottle, right?). It is only the inside of the cap that touches the beer and that gets bathed in liquid when you flip and shake the bottles.
I don't think there is a need to toss the caps in sanitizer. While I sanitize the bottles, I put the caps that I will press on them on top while I shake the <CENSORED> out of the sanitizer in the bottle. That should sanitize the inside of the cap, and leave the outside of the caps fine and dry.