One of the big ideas about magic in his universe isn’t just that the wizards are useless but that using magic is more trouble then it’s worth. It creates all sorts of left over magic residue that can build up to a myriad of effects.

We see the wizards preform powerful spells, showing that they can do have power and do have a certain degree of knowledge, but rather choose not to.

The duty of the wizards is more to make sure no one preforming magic willy nilly and to prevent people from making sorcerers.