But I don’t think even that is the case, as they can essentially just “swap out” the video they’re streaming

You’re forgetting that the “targeted” component of their ads (while mostly bullshit) is an essential part of their business model. To do what you’re suggesting they’d have to create and store thousands of different copies of each video, to account for all the different possible combinations of ads they’d want to serve to different customers.