This 103chapter considers the utility of measuring judicial performance and explores the debate on the use of quantitative and qualitative data in performance measurement. It also presents some performance measurement mechanisms used internationally. Here, judicial performance includes both performance of judges and the performance of the judicial system and the institution.

Why Measure Judicial Performance?

There are general benefits of performance measurement of any organisation. This applies to the judiciary as well: a large volume of information is concretised in clear and concise language, the setting and design of performance targets and improvement plan is simplified, perceptions and biases are eliminated, staff and personnel are incentivised and motivated, and resource allocation and budget decisions are rationalised. I list four broad advantages of measuring judicial performance: three pertaining to performance measurement relating to judges and one pertaining to performance measurement of the system.

Measuring Performance of Judges

Measuring the performance of judges could play a crucial role in judicial appointments. The Supreme Court of India in its latest order in the National Judicial Appointments Commission case (NJAC case) has directed that a fresh memorandum of procedure (MoP) for judicial appointments be prepared by the government in consultation with the collegium.¹ The court has advised that the MoP include eligibility criteria. The importance of quantitative 104data cannot be ignored. It is noteworthy that the National Judicial Appointments Commission Act, 2014 (NJAC Act), now declared ultra vires, specified ‘ability, merit, and any other criteria of suitability as may be specified by regulations’² as the basis for appointment of Supreme Court and High Court judges.

Independent of appointments, assessments of judges are internally useful. According to the American Bar Association’s influential Guidelines for the Evaluation of Judicial Performance, the broad goal of judicial performance measurement is to ‘improve the performance of individual judges and the judiciary as a whole’.³ Apart from supplying useful information in cases of re-election, retention, and other cases of continuation of judges in office⁴ (that is, its role in judicial appointments as described in the preceding paragraph), the guidelines also recognise that the measurement programmes can assist in (a) promoting judicial self-improvement by providing valuable feedback and (b) assignment of judges to rosters as well as improving the design of continuing education programmes.⁵

Performance measures are also a useful way of ensuring judicial accountability without impinging judicial independence.⁶ A judge authoring an unusually low number of judgments or a judge with poor case management skills will be held accountable through well-designed, objective, and transparent evaluations. The line, however, is thin. In April 2001, Wah India, a magazine edited by Madhu Trehan, circulated a questionnaire among 50 senior counsels to evaluate judges of the High Court of Delhi on grounds such as integrity, understanding of the law, punctuality, among others. The assessment was then published with a photo of each judge along with his/her scores. This, however, invited the attention of the High Court of Delhi in a criminal contempt case, forcing Madhu Trehan to issue an unconditional apology.⁷ This is not unique to India — it is a concern shared by judiciaries across national borders.⁸ However, performance measurement mechanisms instituted by judiciaries are likely to assist in shielding it from faulty ones instituted by other branches of the state.

Measuring Performance of the System

The introduction of reforms and the subsequent analysis of the impact of the reforms can be amply informed by judicial performance measures. India ranks a lowly 178 in the ‘Enforcing Contracts’ category of the Doing Business Report prepared by the World Bank.⁹ For this category, the time and cost of resolving a commercial dispute at the first-instance level, as well as a multidimensional ‘quality of judicial processes’ index are analysed. Among the best practices recommended by the study is the institution of specialised commercial division or courts. In response, amendments have been made to the Arbitration and Conciliation Act, 1996¹⁰ and legislation has been passed to constitute commercial courts, commercial division, and commercial appellate division in the High Courts.¹¹ The impact of these reform efforts will be seen in subsequent World Bank reports.

Broadly, the utility of performance evaluation is well summarised by the European Commission for the Efficiency of Justice (CEPEJ): ‘The main aim of judicial statistics is to facilitate the efficient functioning of a judicial system and contribute to the steering of public policies of justice.’¹²

Quantitative and Qualitative Performance Criteria: Notes on a Preliminary Debate

In 2004, Stephen Choi and Mitu Gulati made a radical proposal: they argued for a ‘tournament 105of judges’ for selection of the next Supreme Court Justice in the United States.¹³ Choi and Gulati argued that the most meritorious judge, as identified by objective, measurable and quantitative criteria, ought to be selected as the next Supreme Court Justice. Choi and Gulati ranked judges according to influence (citation counts), productivity (number of opinions and number of cases participated in) and independence (quantitative measurement of willingness to oppose politically like-minded judges). The article led to considerable controversy. Among the first to respond was Lawrence Solum, who argued that Choi and Gulati’s tournament would be unable to identify excellent judges.¹⁴ Solum theorised that excellent judges are those who possess judicial virtues, such as courage, craft, wisdom, skill, among others, and argued that while these virtues are discernable, they are soft, non-quantifiable variables.¹⁵

The Choi and Gulati–Solum debate captures the essence of a larger field of academic deliberation: the utility of quantitative measurements in judicial performance measurement. The thriving nature of this debate is reflected in arguments in the recently concluded NJAC case.¹⁶ Among the three justices described by the Attorney General as examples of bad appointments, one was known to have authored just seven judgments while on the Supreme Court Bench and another was known to be ‘inevitably late in commencing court proceedings’. Some commentators, at the time of argument, remarked that ‘the number of decisions given by a judge is immaterial’.¹⁷

One of Choi and Gulati’s core responses was that their measures (and arguably, other quantitative measures) would not, in absolute terms, identify a great judge, but succeed in marking out relative performance.¹⁸ They used an analogy from the Tour de France: it is challenging to say what intrinsically makes Lance Armstrong a cycling great (this article was written in 2004), but one can very well establish this by comparing his Tour de France wins with those of his fellow competitors. Consider the response of the Supreme Court to the Attorney General’s arguments: ‘He may well be right in his own perception, but the misgivings pointed out by him may not be of much significance in the perception of others, specially those who fully appreciate the working of the judicial system.’¹⁹ In this instance, one could argue in response to the court that quantitative data sheds light on the relative or comparative performance of the judge — while excluding the role of ‘perception’.

This debate is not limited only to performance of judges, but extends even to the performance of the judicial system. The question for example is: must one only take into account the presence of separate small causes courts and the access fees, or must a researcher engage with and survey litigants and court employees?

Considerable debate exists on whether quantitative data should be a conclusive indicator of judicial performance. The International Consortium for Court Excellence (ICCE), for example, points out that there is a ‘worldwide tendency’ to assess court performance using quantitative indicators (such as case clearance rates or cost per case) based on the maxim, ‘justice delayed is justice denied’.²⁰ The ICCE in its report however argues that there may be cases where ‘justice hurried’ becomes ‘justice buried’, thus distorting the full picture for evaluators. Thus it is contended that the relative ease of quantitative measures must not cloud our effort to measure the judicial quality in a broader sense. However, few can deny its value. Evaluation of judicial performance through quantitative data allows for objectivity, nips the role of political, social, and ideological biases²¹ and allows for transparency. In many instances, quantitative data supplements qualitative analysis or may be part of multidimensional indices allowing for robust evaluations.

106Measures of Judicial Performance

This section discusses some measures of judicial performance developed internationally.

International Framework for Court Excellence (IFCE)

The IFCE was developed by an international consortium consisting of the Australasian Institute of Judicial Administration (Australia and New Zealand), the National Center for State Courts (United States), the Federal Judicial Center (United States), and the Subordinate Courts of Singapore.²² The aim of the consortium, through the IFCE, has been to develop a common, universal benchmark and framework, so that courts around the world can engage in self-evaluation and improve the quality of justice administration. The IFCE identifies 10 core ‘Court Values’ based on which Courts may aspire for ‘excellence’: equality before the law, fairness, impartiality, independence of decision-making, competence, integrity, transparency, accessibility, timeliness and certainty. Based on these values, seven specific areas for court excellence are listed: court leadership and management (driver), court planning and policies, court resources, court proceedings and processes (systems and enablers), client needs and satisfaction, affordable and accessible court services, and public trust and confidence (results).

It is important to appreciate the wide range of areas of ‘court excellence’ laid out by the IFCE: often, analysts tend to focus simply on court proceedings and processes, while overlooking other drivers of court performance.

Global Measures of Court Performance

The primary IFCE report, however, does not provide clear and precise performance measures. A parallel document, again developed by the consortium experts, called the Global Measures of Court Performance²³ provides 11 ‘focused, clear, and actionable core performance measures’. These 11 measures are consistent with court values and areas of court excellence. They are as follows:

1. Court user satisfaction: Percentage of court users who opine that court provides adequate procedural justice.

2. Access fees: Average fees collected.

3. Case clearance rate: Percentage of disposed of cases vis-à-vis new cases filed.

4. On-time case processing: Percentage of cases resolved or closed within time reference points, differentiated by case type.

5. Pre-trial custody: Average time for which defendants are jailed, awaiting trial.

6. Court file integrity: Percentage of case files that meet benchmarks such as accuracy, timeliness, probability of time retrieval, etc.

7. Case backlog: Percentage of cases exceeding certain time benchmarks.

8. Trial date certainty: Proportion of trials that are held when first scheduled.

9. Employee engagement: Percentage of court employees who feel that they are productively engaged in the mission and work of the court.

10. Compliance with court orders: Percentage of monetary obligations collected.

11. 107Cost per case: Average cost borne by the court in resolving a case, sorted by case type.

European Commission for the Efficiency of Justice (CEPEJ)

The CEPEJ was instituted by a Committee of Ministers of the Council of Europe.²⁴ Its aim is to improve the efficiency of justice systems of member states²⁵ through tools such as statistical evaluation.²⁶ The CEPEJ issues reports on the European judicial systems with reference to the ‘efficiency and quality’ of justice every two years. These reports are comparative in nature and contain a wealth of cross-country data. The nature and method of the study is worth considering since legal systems of vastly disparate countries across the European continent are considered.

The evaluation process was based on collection of data from national correspondents (mostly based within the respective law and justice ministries) of 45 member-states. The national correspondents respond to a questionnaire comprising of 208 questions, both of quantitative and qualitative nature.²⁷ The questions are organised under 12 heads. First, demographic and economic data is sought, and it includes budgetary data on the judicial systems. The second contains questions on the availability of legal aid and other questions concerning the users of the court (such as rights of court users and public confidence in the system). The third seeks data on organisation of courts, number of judges, number of public prosecutors, management of court budget, internal mechanisms for evaluation of performance of judges, among others. One of the most crucial heads, the fourth, ‘fair trial’ contains questions relating to first principles (presence of procedures to enforce basic rights) as well as questions relating to case flow management and timeframes of judicial proceedings. To elaborate on the latter, the questions primarily seek data on (a) on pending cases at the beginning of the year, incoming cases, resolved cases and pending cases at the end of year and (b) on the length of proceedings in cases, both categorised by the type of cases (for example, homicide, insolvency, divorce, etc.). This enables computation of clearance rate, disposition time, efficiency rate, etc. The fifth contains questions on the service conditions of judges and public prosecutors, including questions on appointment, removal, disciplinary procedures, training, financial benefits, etc. The sixth and seventh heads seek information on the status of the legal profession and alternative dispute resolution mechanisms, respectively. The subsequent heads encompass enquiries on enforcement of decisions, notaries, court interpreters, judicial experts and possible reforms.

European Union (EU) Justice Scoreboard

Based on data from the CEPEJ and supplementary sources such as the World Bank, World Economic Forum, and European Judicial Network, the European Commission releases the EU Justice Scoreboard (‘Scoreboard’).²⁸ The purpose of the Scoreboard is to provide ‘objective, reliable, and comparable’ data on the ‘quality, independence, and efficiency’ of justice systems (specifically civil, commercial, and administrative cases) of the EU member states. The Scoreboard is (a) taken into account while determining EU-level funding priorities and (b) used in making assessments of justice systems under the country specific recommendations (CSRs) issued by the European Council.²⁹ To determine efficiency, the Scoreboard charts length of proceedings, clearance rate, and pending cases. The Scoreboard also charts the time taken for specific cases important for business and the economy, such as insolvency cases, public procurement cases, and competition law cases. To measure quality, the following factors are considered: availability of monitoring-evaluation-survey tools (such as annual 108activity reports, regular evaluation systems, surveys aimed at parties, etc.), usage of information and communication technology, communication policy (such as publishing judgments online, designated officials to explain judicial decisions to the media, etc.), use of alternative dispute resolution methods, training of judges, resources (court budgets, legal aid budgets, number of judges, etc.), and share of female judges. Independence is also measured using perception surveys, and an assessment of structural factors such as the nomination process, powers of judicial councils, method of determination of financial allocation, safeguards against transfer of judges, etc. The Scoreboard also maps justice reform initiatives by the member states.

Concluding Remarks

While questions on methodology, data sources, and impact of judicial evaluation measures remain unresolved, there is broad consensus that periodic judicial performance measurement has significant utility. More and more countries are moving towards having an evaluation system in place. In India, while there is some publicly available quantitative data, it is far from adequate.³⁰ The Supreme Court e-Committee only recently unveiled the National Judicial Data Grid (NJDG),³¹ which provides district judiciary level data on institution, disposal and pendency of cases as well as special information on pending cases instituted by senior citizens and women. There is also a glaring absence of any official³² judicial performance evaluation mechanism. The National Court Management Systems (NCMS) Committee instituted by the Supreme Court for the purpose of establishing a ‘National Framework for Court Excellence’ (setting measurable performance standards) and ‘National System of Judicial Statistics’ has not produced any such framework or common system so far.³³ In this context, data and reports from DAKSH, relating to case clearance rates, case pendency, and case length, among others, are especially handy. This data relates mostly to the performance of judges, but it is nonetheless a useful starting point.

Before closing, it is also germane to note the role of incentive structures and judicial behaviour. Two points, chiefly concerning to the performance assessment of judges, are worthy of close attention. One, as Judge Posner points out in his illuminating essay, the behaviour of judges across different legal systems and within a legal system is likely to differ due to differential incentives and institutional settings.³⁴ Any performance evaluation scheme must take this heterogeneity into account. Second, any official judicial performances have to be carefully crafted, since it has the potential to alter judicial behaviour to the detriment of the rule of law. If the performance evaluation results are highly visible or form the basis of promotions and higher appointments, judges have incentives to game the system and alter behaviour accordingly.³⁵ Any official methodology will thus have to be carefully crafted.

Notes

1. Supreme Court Advocates-on-Record Assn. v. Union of India, 2015 SCC OnLine SC 1322.

2. Ss. 5(2) and 6(3), NJAC Act.

3. Guideline 1-1.

4. Guideline 1-2.

5. Guidelines 2-1 and 2-2.

6. David Brody. 2008. ‘The Use of Judicial Performance Evaluation to Enhance Judicial Accountability, Judicial Independence, and Public Trust’, Denver University Law Review, 86(1): 115–156.

7. Surya Prakash Khatri v. Madhu Trehan, 2001 SCC OnLine Del 590: 2001 Cri LJ 3476.

8. 109Stephen Colbran. 2006. ‘A Comparative Analysis of Judicial Performance Evaluation Programmes’, Journal of Commonwealth Law and Legal Education, 4(1): 35–67.

9. World Bank. 2016. Doing Business 2016: Measuring Regulatory Quality and Efficiency. Washington, DC: World Bank Group.

10. Arbitration and Conciliation (Amendment) Act, 2015 (No. 3 of 2016).

11. The Commercial Courts, Commercial Division and Commercial Appellate Division of High Courts Act, 2015 (No. 4 of 2016).

12. Guideline 1, CEPEJ Guidelines on Judicial Statistics.

13. Stephen J. Choi and G. Mitu Gulati. 2004. ‘Choosing the Next Supreme Court Justice: An Empirical Ranking of Judge Performance’, Southern California Law Review, 78: 23–118; Stephen Choi and Mitu Gulati. 2004. ‘A Tournament of Judges?’, California Law Review, 92: 299–322.

14. Lawrence B. Solum. 2004. ‘A Tournament for Judges. Mad? Brilliant? Clever?’, Legal Theory Blog, 17 April, available online at http://lsolum.typepad.com/legaltheory/2003/04/a_tournament_fo.html (accessed on 15 February 2016).

15. Lawrence B. Solum. 2005. ‘A Tournament of Virtue’, Florida State University Law Review, 32: 1365–1400.

16. Supreme Court Advocates-on-Record Assn. v. Union of India, 2015 SCC OnLine SC 964.

17. Faizan Mustafa. 2015. ‘Allowing Judges to be Judged’, Hindu, 20 June.

18. Choi and Gulati, ‘Choosing the Next Supreme Court Justice’, p. 30.

19. Supreme Court Advocates-on-Record Assn. v. Union of India, 2015 SCC OnLine SC 964. Emphasis supplied.

20. ICCE. 2013. The International Framework for Court Excellence, 2nd Edn., March 2013, p. 29. Available online at http://www.courtexcellence.com/resources/the-framework.aspx (accessed on 15 February 2016).

21. J.K. Elek and D.B. Rottman. 2014. ‘Methodologies for Measuring Judicial Performance: The Problem of Bias’, Oñati Socio-legal Series, 4(5): 863–879.

22. ICCE, ‘International Framework for Court Excellence’. The ICCE was assisted by other expert bodies such as the European Commission for the Efficiency of Justice, the World Bank and Spring Singapore. The ICCE now has additional members from numerous countries.

23. The latest available is Discussion Draft Version 3, dated 9 November 2012, prepared by Dan Hall and Ingo Keilitz.

24. Resolution Res (2002) 12, Adopted by the Committee of Ministers on 18 September 2002 at the 808th meeting of the Ministers’ Deputies.

25. Art. 1, CEPEJ Statute.

26. Arts. 2(1)(a) and 3, CEPEJ Statute.

27. Scheme for Evaluating Judicial Systems, 2014–2016 cycle, CEPEJ.

28. European Commission. 2015. The 2015 EU Justice Scoreboard, p. 5. Brussels: European Union. Available online at http://ec.europa.eu/justice/effective-justice/files/justice_scoreboard_2015_en.pdf (accessed on 15 February 2015).

29. European Commission. 2015. ‘The 2015 EU Justice Scoreboard: Questions & Answers’, 9 March, MEMO/15/4576, available online at http://europa.eu/rapid/press-release_MEMO-15-4576_en.htm (accessed on 15 February 2015).

30. Nick Robinson. 2013. ‘A Quantitative Analysis of the Indian Supreme Court’s Workload’, Journal of Empirical Legal Studies, 10: 570–608; Aparna Chandra. 2015. ‘The State of Judicial Statistics in India’, Daksh Blog, 10 February, available online at http://dakshindia.org/the-state-of-judicial-statistics-in-india/ (accessed on 12 May 2016).

31. The NJDG is available online at http://164.100.78.168/njdg_public/main.php#.

32. Barriers such as the threat of contempt of court actions remain disincentives for private actors to design unofficial measures. It is recommended that any proposal to have an official system in place must be statutorily or constitutionally backed to prevent undue interference from branches of government.

33. According to Supreme Court Office Order No. 4/SG/NCMS/2012, dated 2 May 2012, the NCMS was established under the supervision of the Chief Justice of India and with the institution of a 11-member committee.

34. Richard A. Posner. 2005. ‘Judicial Behavior and Performance: An Economic Approach’, Florida State University Law Review, 32: 1259–1279.

35. Solum, ‘A Tournament of Virtue’.