Indian Kanoon - Search engine for Indian Law

6. In regard to merits, the Commission contended that the 'statistical scaling' method adopted in regard to Civil Judge (Junior Division) Examination is legal, scientific and sound and its policy to apply statistical scaling to marks of written examination, was based on experts' opinion as also the experience gained in conducting several examinations. It is submitted that under the proviso to Rule 50 of the U.P.Public Service Commission (Procedure and Conduct of Business) Rules, 1976, it is entitled to adopt any formula or method or device to eliminate variation in marks; that it found variation in the marks awarded by different examiners on account of a phenomenon known as 'examiner variability' and to eliminate it, statistical scaling was introduced. It is further submitted that matters relating to the conduct of Examination, evaluation of answer-scripts, application of methods to bring in uniformity in evaluation are matters of policy involving technical and scientific decisions based on expert opinion; that courts are not equipped to pronounce upon such matters and, therefore, should not interfere in the absence of manifest arbitrariness or mala fides; and that, at all events, in the absence of an opinion by a body of experts in the field of statistics certifying that the system of scaling adopted by the Commission is unsound and irrational, there should be no interference. Lastly, it is submitted that if the court, for any reason, should hold that the existing scaling system should be substituted, that should be done prospectively.

The above procedure of 'moderation' would bring in considerable uniformity and consistency. It should be noted that absolute uniformity or consistency in valuation is impossible to achieve where there are several examiners and the effort is only to achieve maximum uniformity.

24. In the Judicial Service Examination, the candidates were required to take the examination in respect of the all five subjects and the candidates did not have any option in regard to the subjects. In such a situation, moderation appears to be an ideal solution. But there are examinations which have a competitive situation where candidates have the option of selecting one or few among a variety of heterogenous subjects and the number of students taking different options also vary and it becomes necessary to prepare a common merit list in respect of such candidates. Let us assume that some candidates take Mathematics as an optional subject and some take English as the optional subject. It is well-recognised that a mark of 70 out of 100 in mathematics does not mean the same thing as 70 out of 100 in English. In English 70 out of 100 may indicate to an outstanding student whereas in Mathematics, 70 out of 100 may merely indicate an average student. Some optional subjects may be very easy, when compared to others, resulting in wide disparity in the marks secured by equally capable students. In such a situation, candidates who have opted for the easier subjects may steal an advantage over those who opted for difficult subjects. There is another possibility. The paper setters in regard to some optional subjects may set questions which are comparatively easier to answer when compared some paper setters in other subjects who set tougher questions difficult to answer. This may happens when for example, in a Civil Service examination, where Physics and Chemistry are optional papers, examiner 'A' sets a paper in Physics appropriate to a degree level and examiner 'B' sets a paper in Chemistry appropriate for matriculate level. In view of these peculiarities, there is a need to bring the assessment or valuation to a common scale so that the inter se merit of candidates who have opted for different subjects, can be ascertained. The moderation procedure referred to in the earlier para will solve only the problem of examiner variability, where the examiners are many, but valuation of answer scripts is in respect of a single subject. Moderation is no answer where the problem is to find inter se merit across several subjects, that is, where candidates take examination in different subjects. To solve the problem of inter se merit across different subjects, statistical experts have evolved a method known as scaling, that is creation of scaled score. Scaling places the scores from different tests or test forms on to a common scale. There are different methods of statistical scoring. Standard score method, linear standard score method, normalized equi- percentile method are some of the recognized methods for scaling.

(iv) If scaled score after scaling is more than maximum marks, then candidate will be allotted maximum marks in the said group/subject.

29. Eversince then, the Commission has been following the statistical scaling. According to the Commission, the scaling method is rational, scientific and reasonable and would lead to assessment of inter se merit of the candidates in a just and proper manner. The use of the said method was reviewed by an Expert Committee on 31.7.2000 and it was reiterated that the formula and method presently used for scaling can be continued to be used in future also and there was no need to change the same. Thus the scaling is continued.

All the three cases related to moderation and not scaling. There are, however, passing references to scaling as one of the methods to achieve common standard of assessment. The fact that scaling is a standard method of assessment, when a common base has to be found for comparative assessment of candidates taking examinations in different optional subjects, is not in dispute. In fact the Commission may continue to adopt the said system of scaling, where a comparative assessment is to be made of candidates having option to take different subjects. The question is whether scaling, in particular, linear standard scaling system as adopted by the Commission, is a suitable process to eliminate 'examiner variability' when different examiners assess the answer scripts relating to the same subject. None of the three decisions is of any assistance to approve the use of method of 'scaling' used by the Commission.