It's a bunch of things. Cathedrals have canons, who are resident clergy, but not all clergy who work at cathedrals are necessarily given this distinction. Bishops have canons who work for/with them, and attaching a canonry (yes, that's the word) indicates the bishop's high regard for and confidence
in the person. Finally, there are honorary canons, clergy who have distinguished themselves in some ways or on whom the bishop relies for special advice or other "services to the don." What it boils down to, especially in the last two cases, is the Anglican equivalent of Monsignor ("domestic prelate"!). We have also had three lay people in the status last year.

Long Answer--The Rev. Canon A. Malcolm MacMillan:

People often ask me that question when they learn that I am one.  My usual answer is that we are 
people in the Church who like to think we are big shots, when in fact we are often nothing but big bores.  If the inquirer wants a more serious answer, I usually go on to say that in the Anglican Communion canons are somewhat comparable to Roman Catholic monsignors or Eastern Orthodox archimandrites, but that is only partially true.  All three titles are usually reserved for senior clergy who have been recognized for distinguished service to the Church, but there have also been a few lay canons in this Diocese and elsewhere in the Anglican world. And it is also true that many canons bear the title in recognition of leadership responsibilities they bear in the church, usually as members of a Bishop's staff, or most commonly, as members of a Cathedral Chapter.  I understand that monsignors are considered to be members of the Pope's household and are appointed by him. Archimandrites are usually senior priests with administrative responsibilities, often monks who are heads of  monasteries who as monks are eligible for election to the episcopate. I like to think of canons as members of the Bishop's household, either because they are on his staff, or on the staff of his cathedral or honored by him for some special reason. In the American church and elsewhere, it is customary for canons to 
wear purple cassocks (as do cathedral choristers and acolytes), purple being the bishop's "livery" color and there-fore worn by his staff and members of his household, but I have been told that this is not the case in England where each cathedral chapter has the right to establish its own livery color. The Cathedral of Chichester, for example, has gray as its livery color, and many royal "peculiars", i.e. churches, chapels and some cathedrals under royal patronage wear red. Blue is quite popular and plain old black is not uncommon. If ecclesiastical haberdashery is your thing, you should get a catalog from Whipple's and see what they advertise for canons: Black cassocks with red piping, fancy lace surplices and shoulder capes trimmed with rabbit's fur! There is a fascinating chapter on "canons regular" in C. H. Lawrence's "Medieval Monasticism" published by Longman, London and New York, page 163 
ff.  It goes into the origins of the office of canon in great detail, pointing out that the establishment of chapters of canons was part of the eleventh century reforms undertaken by the Gregorian papacy to bring secular clergy, especially those who were married and serving cathedrals, into the pattern of the religious orders. Members of the monastic orders owed obedience to the superiors of their orders (usually resident in Rome) whereas canons were under the patronage of their bishops.  In Scotland,
where King David I undertook to force the clergy of the Celtic Church to convert to Rome, monks of the Celtic monasteries were allowed to reconstitute themselves as chapters of canons under the diocesan bishops whose sees were set up by the crown. It should be noted that Celtic monks were not required to be celibate, but those who became canons regular were required to put aside their wives and families and to adopt the so-called Rule of St. Augustine for Canons Regular.  In England, 
the chapters of canons survived Henry VIII's suppression of the monastic orders, probably because they were not directly under superiors in Rome, but under English bishops almost all of whom supported the break with Rome, and because they wre necessary to the life of the English Cathedrals. In theory, all bishops in the Church of England must be elected by the Cathedral Chapter of the diocese to which the crown nominates them. I seem to remember that William Temple became Archbishop of Canterbury because the canons of Canterbury refused to elect anyone else.  As far as I am aware, there are no national canons (i.e., laws or rules) which define the office of canon 
(Literally "one living under a rule,") in the American Church and so the office or position is largely honorary, although vestiges of the original character of the position survive in some cathedrals actually governed by chapters of voting canons (somewhat like vestries). Personally, although I feel greatly honored that Bishop Rowley made me "Canon Pastor" of the Diocese of Northwestern Pennsylvania "in recognition of twenty nine years of faithful service as a devoted pastor to both the laity and the clergy of the Diocese" I find it difficult to understand why one should be honored for doing that for which one has already been so richly rewarded by the kindness and friendship of so many wonderful people. 
 
 

Right Answer-- Canon Bill Llewellis:

My early Roman Catholic training suggests this answer: It is a mystery. One
stands at the edge in awe and wonder. :)