Fighting a well-played mage is hard for almost any character. With the abundance of control and their ability to be where they need to be, they are a useful to say the least. While it is important to maximize your ability with your character, knowing a few weak points is critical for killing a mage.
For some classes the opening move is the most important. Without a doubt, this is the most important point for many fights. In a duel this will likely be a polymorph from the mage simply to give him a chance to Spellsteal your buffs. This is done to shift the buff focus strictly to his favor, while leaving you unbuffed. He is probably going to be spellstealing you anyways, so it is important to not use your trinket at this time. Using it here will almost certainly spell defeat for you as he can line up a large shatter combo to kill you fast without fear of you breaking from it.
Mages will often flaunt their superior maneuverability by weaving in and out of melee range keeping you frozen while dropping back to hit you with his spells. They are able to keep the pressure on the move with Ice Lance even if it is a weaker damage spell. The goal to kill a mage is to limit, or eliminate, this maneuverability.
A mage that is stuck in one place is as good as dead. If you can keep them put for any significant amount of time, you should be able to kill them. This is doubly as true if you are a melee class.
It is worth saying that you will have to get the mage to use their trinket at some point before you can kill them. This is a difficult proposition simply because blink can get the mage out of many trouble spots already. If you can make the mage blow his trinket and catch him with blink on cooldown, you will have a much easier time taking him out.
If there is one thing you should really watch for, it is for Ice Block. They are going to use this to keep them alive for at least another round of minor cool downs. In most cases this will be followed by a Blink as they are expecting you to focus your attention on the area around the block. Save your gap closers for this time and you can have them in a better spot.
Overall, mages are some of the hardest encounter you will face in PvP. They seemingly have a counter for nearly everything and are able to live long enough to give any class trouble. The easiest solution is to focus your group's attention on the mage long enough to kill them, but barring that these tips will help.