Quotations for today:
“Criticism, though dignified from the earliest ages by the labours of men eminent for knowledge and sagacity, has not yet attained the certainty and stability of science”
Samuel Johnson (creator of the first English dictionary)
“Almost all our knowledge is only probable; and in the small number of things that we are able to know with certainty, in the mathematical sciences themselves, the principal means of arriving at the truth—induction and analogy—are based on probabilities. I believe that we do not know anything for certain, but everything probably” Christiaan Huygens (17th Century astronomer)
“Objective evidence and certitude are doubtless very fine ideals to play with, but where on this moonlit and dream-visited planet are they found?” William James (19th century psychologist and philosopher) The Will to Believe
”The demand for scientific objectivity makes it inevitable that every scientific statement must remain tentative for ever”. Karl Popper (philosopher of science) The Logic of Scientific Discovery
Music exams also provide a sorely needed response to another set of issues.
The mixture of a strong inclination to risk aversion, the desire to protect and justify one’s actions, and an absolute obsession with targets and measurability, has given educators (and many others) an increasing nightmare of forms to be filled, and boxes to be ticked. A friend recently took early retirement from teaching, having decided that an impoverished life was preferable to the depressing morass of paper work – assessments, monitoring, reports, statements of aims which had come to mean that she spent more time on such matters to meet the demands of professional administrators than she did on teaching. She seems to be one of a large number of teachers who would at least like to follow the same route.
The latest lunacy of which I have heard is the suggestion that computer marking of examinations will become universal within a decade or so. This is pure sci-fi. The only way in which a machine such as a computer can do this is by reading answers entered (as for example in certain IELTS papers) into boxes into computer readable form (e.g. Y or N, 1,2,3 etc). The IELTS authorities apply this to the Reading and Listening Tests, which are carefully devised to produce answers which can be rendered into such a form. But the same authorities continue to use examiners (human beings with ears, eyes and a brain) to mark the Speaking and Writing Tests. Why ? Because there is no sign that computers, even by using Artificial Intelligence, are anywhere near able to do (even with effort) what humans are supremely equipped to do, which is to assess what they experience according to (often complex) criteria.
Musicians have long recognised this, because it is in the nature of the subject. let’s consider the case of a Grade exam in Piano Playing (the same applies to other instruments).
Normally the student must demonstrate competence in the “nuts and bolts” of music – such as scales, chords, and in the Aural Test, the recognition of the interval (distance) between two notes. These requirements can defined precisely. In the exam a named scale can be requested, to be played at a named speed, volume, and articulation (e.g. staccato). These elements are susceptible to precise measurement, and it would be possible to devise a system which identified every departure from the stated criteria. In fact, this is not done. It would be very costly to create and administer, and I doubt that there would be sufficient improvement on the judgement of a trained examiner, to merit the expense.
The student is also required to perform various pieces (typically three with contrasting natures and problems). These will embody those aspects of technique, style and expression which the Examiner Panel (consisting of experienced examiners) has deemed to be a natural progression logically and musically, and which are also appropriate to a given level of study. Thus, for example, the performance of a Bach fugue would only be expected at a high grade (7 or 8), whereas a Two Part Invention would be a more suitable move into polyphonic music at, say, Grade 5.
It will be useful for some readers if I give an example from a very different type of exam – the Electric Guitar exams of the Registry of Guitar tutors. These also require demonstrations of technical ability. But since improvisation is an essential skill for any player in non-classical areas such as pop, rock or jazz, the ability to improvise is also tested from the earliest stages. This is not the place to go into technical detail, but I can attest from personal experience of the Examiner training sessions, that what counts as competence is very clearly demonstrated in detail (giving for instance the notes which are or are not to be expected, and the types of rhythmic flexibility which a candidate should show). The standards are very clearly articulated, not least because the Registry gives high importance to using examiners who are or have been working musicians, but the latter often expect standards which would be unreasonable demands on the typical learner.
The above examples show that again, the demands of the pieces can in some respects be stated in terms of precise criteria, and are measurable. The same point about the cost of mechanical assessment applies.
However, it is obvious that music is not only or even primarily a matter of definable techniques, and in this it again presents an illustration of what can, and I assert has to hold in a sensible examination system.
The good performer must exhibit an increasing grasp of expression, of the emotional dimension of music, of the nature of authenticity and appropriateness within a given style, and in the “popular” forms, a sense of the precision and movement of rhythm parts.
Again, these can be defined, but they cannot easily be measured. Nor do they need to be, because they can be perceived by a trained listener. At this point, the exam system needs to stop a compulsion to seek out some spurious and in fact unattainable objectivity, but to accept that we are and have to be in the realm of subjective human judgements. A judgement on the “conviction” manifested in a given performance is an opinion, not a measurement.
But this is not to say that any old emotional reaction by the judge will do, that anything goes, and that the candidate’s mark is a matter of chance – which examiner you get – and an arbitrary judgement – whether the examiner happens to like what you do.
This is because the examiner has to be selected by peers, who assess whether his/her understanding of what is to be assessed is indeed professional and in line with what is generally accepted. The latter phrase may horrify some people. They may argue that this allows the existence and power of self-selecting clubs, which may well be out of touch or even opposed to the values of another group.
If by this they mean that the examiners of the Associated Board are not typically interested in, responsive to or capable of judging Heavy Metal music, they are probably right. But the “AS Board” examiners don’t do this, they don’t seek to, and make it quite clear that their exams are not concerned with this sphere of music. By contrast, the Registry Electric Guitar panel does contain such experts (and they are experts).
My point is that in any given area (not just music) there is a body of agreed assumptions and values, which can be stated, monitored or passed on. Of course, once in a lifetime you may have in front of you the next Jimi Hendrix or Charles Ives, who presents you with a musical experience which puzzles, which you do to know how to assess. But exam systems are not there to detect originality and genius, but to assess that widely agreed body of knowledge which the overwhelming mass of the population needs to master. Geniuses have necessarily to make their own way.
By now, many readers may be anxious that such a system is nevertheless open to error or even abuse. I understand this fear. I have been at many standardisation meetings in several spheres, and sometimes the range of marks awarded in the training sessions is worryingly wide. Added to this there is clearly the danger of human error, or (much more rarely) prejudice.
As the matter is ultimately subjective, there must always be some risk. There is and can be no objective truth, no 100% certainty in such matters.
But the danger can be and is minimised in various ways. Standardisation tests (of the examiners), moderation, second marking and other checks are routine. Statistical checks can reveal if there are departures from the norm, which can then be investigated. There is an appeal procedure, to give a further option to candidates who feel that they have suffered some injustice.
The most obvious way to counter the dangers of subjectivity is to increase the numbers of examiners – at a certain point the consensus becomes clear. In practice it is a choice between having one or two. Thus the Cambridge First Certificate in EFL uses two in the Speaking Test; IELTS uses one, backed by recordings which are used for monitoring standards and dealing with appeals. Grade examinations in Music use one, Diploma exams two.
What those who want certainty must realise is that (a) machines won’t do the job (b) it depends how much you are prepared to spend (c) there isn’t any absolute certainty in such areas of knowledge. I believe that point (b) is the one which needs constant repetition, but that politicians don’t say it, for fear of unpopularity. It’s really very simple: you get what you pay for and you pay for what you get. Since there is always a limit on resources, it is the taxpayer who has to decide what the priorities are. Mine are quite simple: health, education, pensions and support during sickness or unemployment. If that means that other goodies are much scarcer, then so be it.
In the present context of music exams I think the level of guarantee of quality is about right, but that there are probably some areas which could be improved. To have four examiners instead of one would offer more guarantee (though it would be daunting for the candidates !) but not such as would merit increasing the administrative costs fourfold.
Today’s message: Music exams show us that in making decisions about assessment we must consider the nature of the knowledge we are dealing with. In the case of Music, (and many other subjects) some aspects are suited to accurate measurement, whereas others need judgement. This inevitably gives a grey area of assessment.