Does this setting actually work: "Send a confirmation email when I receive a new private message"?

Yes, I can say that works (sometimes I wish it wouldn't) but, based on your comments in a Topic over in Help .. I'm not sure you're asking the question you think you are. The Personal Message (PM) is someone sending you a PM via the "Send a PM" link within this Forum application. Whereas, I thought your question was more about the "tracking" feature of receiving a note if someone posted into your Topic, which is a bit different.

Moderating actions of Moving or Merging posts to keep things on-topic seems to change pointers involved with the "tracking" .. so it seems to work "unless" .... I try to notify the poster (via PM) that I've done something to a posting (or Topic) to address this little issue. And yes, I've missed a few of those notifications apparently.