+1 on this.
A hover text or the icon closer to the actual highlight would be a more visually pleasing solution.
This is especially important for us when putting extra effort in making the rich text fields as beautiful as possible (e.g. onboarding documents/documents shared with clients/wiki etc).
As it is right now, the litte icon to the side is a distraction from what is usually a less important link if you look at the document as a whole. Although it really does get your attention since it stands out so much so I understand why it’s like this.
Right now I’m avoiding referencing other entities with a link and instead mentioning the entity directly whenever possible to get around this visual representation.