Posted by: Stephen Forward20 Jul 2014
It’s hard to imagine Jancis Robinson doing a stand-up routine at London’s Comedy Store isn’t it? Yet, after reading her take on the recent Master of Wine (“MW”) examination papers it would seem to me that she has all the hallmarks of a good humorist.
There was one routine in particular that I found amusing where she published the latest MW exam papers on her website, and then commented on the questions each MW student had to grapple with.
While referring to the MW exam questions, Robinson wrote:
“I should imagine many wine lovers will read them and think, 'I could answer those'. But I can assure you that the questions always look deceptively straightforward. The examiners are looking for extremely detailed responses that the questions don't really hint at.”
Amusing indeed, at least it would be if she was joking.
Let’s just read between the lines and understand what Robinson is really saying. The first point: “I should imagine many wine lovers will read them and think, 'I could answer those'”.
From that we can assume the questions were fairly straightforward and, indeed, Robinson admits many people could answer the questions. Great! So loads of people should pass the MW exam then, right? Wrong!
She goes on to say: “But I can assure you that the questions always look deceptively straightforward”.
So what are we to understand from that? Only one thing, really -- the questions are designed to deceive. Why is that? Why should an examiner wish to mislead a candidate with a question? I thought the whole purpose of a question was to be clear in its intention. The definition of Question is “a sentence worded or expressed so as to elicit relevant information”. It says nothing about being deceptive, so why would the MW examiner wish to be deceptive?
But wait, Robinson goes on to explain why the question is purposely misleading: “The examiners are looking for extremely detailed responses that the questions don't really hint at”.
Put another way, Robinson is saying the examiner is looking for a detailed answer to a question that has little or nothing to do with the actual question they are asking!
It would seem that not only do students need to know every finite detail about every aspect of wine but, to qualify as a Master of Wine, candidates are required to be mind readers too. Well that explains everything doesn’t it? The Institute of Masters of Wine (“IMW”) makes Harry Potter’s time at Hogwarts look like a walk in the park.
This old-school approach to an exam, regardless of the subject, demonstrates profoundly that the IMW is an organisation trapped in the dark ages. This out of date attitude reinforces the point of view that the MW exam is totally irrelevant to a modern day wine industry.
Ask any major educational exam body in the world today, and they will tell you that the accepted standard for setting exam questions requires that exams adhere to a number of fundamental principles. Two of the basic principles include:
1) Clarity of the questions; and
2) Fairness in the examiners’ approach.
The MW exam clearly fails on both of these principles.
The IMW fails by Robinson’s own admission on clarity because, as she so eloquently put it, the exam questions are designed to deceive. Moreover, by virtue that “the examiners are looking for extremely detailed responses that the questions don't really hint at” the exam fails the fairness test too.
Examiners must give candidates a fair chance to demonstrate their knowledge and give students the chance to succeed. If examiners ask one question, but really want the answer to another question they’re not even asking, how can that be objective? How can that be fair and just? It is certainly not being clear!
The IMW website contains several videos with post-mortems on past questions indicating what the examiner is actually looking for, and it is interesting to see the length of explanation the examiner goes to in order to explain the question. What is most revealing is that the explanations in many cases bear little resemblance to the brief question posed in the exam. This indeed confirms Robinson’s assertion that the examiner is giving little away in terms of what response he or she is seeking to solicit in a textbook answer.
The examiner says several times in the videos “It’s not rocket science”. Dammed right it isn’t! Yet the IMW appear to go to great lengths turning a fairly straight-forward subject of wine into rocket science. Indeed, there are fewer MWs in the world today than there are qualified rocket scientists.
The exams appear less concerned with the knowledge students have, but rather more focused on the students’ ability in decrypting and second guessing the vague questions posed.
The crunch of the matter is, the examiners are looking for very specific detailed answers to extremely vague questions. That is not a true test of a student’s knowledge, and that is not the real purpose of any exam worthy of respect -- nor should it be.
Given Robinson’s mind-set, which presumably reflects that of the Institute in general, coupled with the indisputable evidence that only 366 people worldwide have ever passed the exam in its sixty year history, what more evidence does one need in order to conclude that the Institute of Masters of Wine is an old fashioned, irrelevant organisation, not fit for purpose?
While the IMW openly publishes the number of candidates that have passed the exam, they appear less open about pass rates and indeed the number of candidates that have failed the exam or dropped out prior to completing the course. I suspect this number is substantial and would likely prove embarrassing for the organisation.
The IMW is little more than an elitist club, accessible by invitation only, designed to keep the riff-raff and rabble out. But other than being a club where members can go off on various jollies comparing each other’s tasting notes, I see no useful or meaningful purpose to the organisation whatsoever. Indeed, the institute is not relevant to a modern day wine industry. It has certainly failed in its original remit to “improve the standard of education within the British wine trade”.
The proof of this is in the job market. There are almost no jobs where the MW qualification is a requisite, hardly surprising given the insignificant number of candidates that have ever qualified in its history. So what purpose does the exam have?
While there is no question the IMW has an abundance of talented wine specialists, what it appears to lack are qualified educationalists. This state of affairs may go some way to explaining why the institute has created a pretty meaningless examination system. While the IMW website boasts an external education advisor, I cannot find evidence that the people who create or mark the exams are true educationalists.
When creating an educational system, such as the IMW, it is critical to have qualified expertise on modern day educational systems, theories and concepts. Indeed, these skills are arguably as important, if not more so, than expertise on the subject matter in question. But ideally, any educational body should have a good balance of all the appropriate skill-sets required.
The IMW ship appears to be overloaded with wine theorists, but limited or no real educationalists to speak of. If the captain in charge of this small vessel cannot counter balance the crew with adequate educationalists, she may as well scuttle the ship altogether for all the good it can do in its current form.
Founder and owner of Essentially Wine Ltd established in 2004, and specializing in importing and retailing fine wines from small producers. Prior to that I spent 20 years in the IT industry, the last 13 or so living and working in Switzerland running a computer services company.