Now it is possible to unmention a tweet on Twitter. If a user mentions you in a tweet from this feature, you can remove that mention yourself.
This feature has not been officially released and now this feature is under testing.
Mention alert will be removed only if the text mentioned in this feature remains. This is a feature to be brought under the anti-harassment safety mode.