I am just starting to use comments heavily on a new project with a new collaborator, and man, it is not (yet) a great experience, especially with the notifications/activity links! I am definitely in support of this idea here and/or at least the activity link going to the specific comment on the page and opening it so I can see the context and reply. As it is it just takes me to the page and I have to find (among 20+ different comment threads) where that reply is so I can deal with it. Very frustrating.
Correction/update: the links in the Notifications stream don’t work only when you are already on the page being linked to. Which in itself is a highly annoying issue. I’ve filed a bug here: Issues with usability of Notifications for in-line comments