ISSN (Print) - 0012-9976 | ISSN (Online) - 2349-8846

A+| A| A-

Dealing with a Deteriorating Statistical Base

India's offi cial statistical collection machinery has been in decline for more than two decades. The National Statistical Commission was supposed to reverse the decline, but the NSC has had to suffer the government's indifference. It is time the users of information, the academic community in particular, wake up to the fall in quality of Indian data. As a fi rst step, academics and public intellectuals should come together to form an India Database Forum which could discuss and highlight what ails the statistical system and then pressure the government to fi x what was once the envy of the world but is now among the weakest.

PERSPECTIVES

Dealing with a Deteriorating Statistical Base

S L Shetty

India’s offi cial statistical collection machinery has been in decline for more than two decades. The National Statistical Commission was supposed to reverse the decline, but the NSC has had to suffer the government’s indifference. It is time the users of information, the academic community in particular, wake up to the fall in quality of Indian data. As a first step, academics and public intellectuals should come together to form an India Database Forum which could discuss and highlight what ails the statistical system and then pressure the government to fix what was once the envy of the world but is now among the weakest.

The author wishes to thank K Kanagasabapathy for his comments on an earlier version of this article. He also wishes to thank Y V Reddy for encouraging him to pursue studies related to the theme of deterioration in the data standards and what could be done to arrest the decline. This article is a revised version of a presentation made at a recent seminar of the Indian Association for Research in National Income and Wealth held in Puducherry during 15-16 March 2012.

S L Shetty (slshetty@vsnl.com) is at the EPW Research Foundation.

I
ndia’s pride in being a nation with a strong base in statistics is getting increasingly punctured. Now for about three decades, particularly after T N Srinivasan exposed the frailty of the Indian statistical system to the international audience in a seminal paper in the Journal of Development Economics (1994), there have been a series of studies expressing concern about the deteriorating quality of the Indian database in general. After surveying the quality of data availability for different sectors of India, Srinivasan (1994: 23-24) had then characterised the situation as “disturbing”. He wrote:

The disturbing conclusion emerging from the discussion in the previous sections is that the situation with respect to the quality, coverage, inter-temporal and international comparability of published data on vital aspects of the development process is still abysmal in spite of decades of efforts at improvements (pp 23-24).

It is widely known that the system developed further cracks during the 1990s after liberalisation, how the problems attracted attention in non-offi cial as well as offi cial quarters,1 and how these finally culminated in the appointment of the National Statistical Commission (NSC) in January 2000. Even though it faced some critical comments including that from T N Srinivasan himself (2003),2 there is no gainsaying that the report of the NSC (2002) was a path-breaking, historical document geared to improve the Indian statistical system quite significantly. The report was unique in the sense that never in the past had there been such a comprehensive study covering all aspects of the statistical system – the institutional and administrative structures, the extent of data gaps in every ministry and department of the Government of India and recommendations to improve them including the suggestions for 10 census studies and about 60 types of sample studies.3

Alas, except for the initial one-shot effort to setting up the NSC in 2005 albeit as a non-statutory and non-permanent body and the provision of staff incentives in the form of upgradation of posts and promotions for the statistical cadre, many substantive recommendations of the NSC concerning improvements in the system of data collection and the quality of data themselves remain to be implemented. The NSC has shown some results: it has got the Indian Statistics Act 2008 enacted, and adopted the Collection of Statistics Rules 2011; it has advanced fairly substantially in defi ning the nature of “core statistics”; it has drafted a national policy on offi cial statistics as also on the National Statistical Code; and has just got published a committee report which has looked into legislative measures on statistical matters. It has also completed a major report on unorganised sectors statistics. While all of these are very useful for laying a fi rm legal and legislative basis for data gathering and dissemination, the substantive questions of improvement in data collection and data quality in different areas remains to be addressed.

A Low Priority

All indications suggest that collection of accurate statistics has a low priority in policymaking today and the intellectual community which studies India’s economic problems also shows no concern for the deteriorating quality of the Indian database, which they otherwise studiously use for economic analysis or for various econometric exercises. The latter then lead to loaded judgments that are based on a weak statistical foundation! A telling example of the scant regard for quality economic and social statistics is to be found in the fact that India has no statistics on the pattern of income distribution but scholars in a nonchalant manner use household expenditure distribution as a surrogate for income distribution. What colossal errors of judgment!

Economic & Political Weekly

EPW
may 5, 2012 vol xlviI no 18

PERSPECTIVES

Undoubtedly, the malaise in India’s statistical system is deep-rooted. A cursory survey suggests how data problems and data weaknesses are widespread and serious. These relate not only to the informal sectors of the economy where the statistical problems are indeed daunting, but also to each and every segment of the organised sector. Corporate sector statistics, industrial statistics comprising, in particular, the index of industrial production (IIP) and the Annual Survey of Industries (ASI), and the macroeconomic data on the capital market – are all in a shambles. Studies show that IIP and ASI grossly underestimate both the level and the annual growth of the manufacturing sector in India. The extremely poor database on the small-scale sector (or now the medium and small and microenterprises – the MSME) is widely known despite periodic nationwide surveys. They are not covered in any case in the all-India IIP. The National Accounts Statistics (NAS) which is otherwise technically robust because it follows the UN System of National Accounts (UN-SNA) is facing flaws because of the defects in its components. Apart from poor industrial statistics, many components of saving and investment estimates are admittedly faulty, partly because of the defects in estimates of corporate savings and investment and to a certain extent because of the weak estimates of rural household saving and capital formation due to deterioration in the sample sizes of the All-India Debt and Investment Surveys (AIDIS). We do not have any dependable estimates of investment in agriculture or any other major sector of the economy. Many quantity and price index number series are dated; and numerous components of the service sector estimates remain guesstimates.

The most innovative method of conducting an economic census of enterprises and their follow-up surveys has failed to produce the desired results such as the creation of a business register. There is no up-to-date information on the number and profiles of non-governmental organisations operating in the microfi nance (mFI) sector. Similarly, there is no reliable data on non-banking fi nancial companies (NBFCs) and their operations. Rural cooperative sector credit and deposits data are four to five years old. For over 25 years now the Central Statistical Offi ce (CSO) has been assuming one-third of the gross/net value added in the organised financial sector as the corresponding estimate for the unorganised nonbanking financial enterprises (moneylenders, indigenous bankers, capital market players, etc).

The Government of India has ceased to publish data on Income-Tax Revenue Statistics after 1999-2000, thus depriving us of opportunities to undertake income distribution studies on urban households. And, fi nally, despite repeated suggestions, no integrated survey has been done on household income, expenditure and savings so that instead of

CALL FOR PAPERS 6th International Conference on Decision Sciences for Performance Excellence

December 27, 28 and 29th 2012 IBS Business School, Hyderabad

The Indian Subcontinent Region Decision Sciences Institute (ISDSI) and IBS Hyderabad are organizing the 6th annual international conference of ISDSI at Hyderabad, India from 27th to 29th December 2012. We invite all Academicians, Practitioners from Business and Industry and Research scholars to this international conference to share, discuss and deliberate on a variety of research agendas, ideas and findings in all disciplines relevant to decision making, decision processes and their impact on performance excellence.

Best Paper Awards:

The conference topics/themes include but not limited to:

The best paper in each category (Theoretical/Empirical, Application,

  • Global Supply Chain Management student paper/Ph.D thesis) will be awarded with a cash prize of
  • Operations Management and Operational strategy Rs. 10,000/-. Selected papers will be published as refereed conference
  • Behavioral aspects of Operations Management proceedings.
  • Risk and Disaster Management
  • Innovative applications in Modeling and Decision techniques Conference Chair:
  • Financial Risk Management Dr. Ravi Kumar Jain, IBS Business School, IFHE University
  • Issues in global banking Prof. Jatinder (Jeet) N. D. Gupta, The University of Alabama
  • Global Investment decision making and M&As Prof. D K Banwet, Dept. of Management Studies, IIT Delhi
  • Capital markets, trading & portfolio management
  • Performance and Revenue Management Conference Co-Chair:
  • Accounting, Auditing and Compliance Prof. Bhimaraya Metri, Strathclyde SKIL Business School
  • Trends and challenges in Family Business Management Prof. Ramachandran Natarajan, Tennessee Tech University
  • Innovations and Entrepreneurship Dr. Sashikala, IBS Business School, IFHE University
  • Innovations in Information Technology applications
  • Decisions Sciences in governmental and social sector organizations
  • Industry sector-specific decision making & Strategy formulation
  • Strategic HRM, Culture and behavioral impact on performance
  • Product, Services and Knowledge management
  • Paper Submission: Submit your papers online at the conference

    website (see URL below) or send your papers to isdsi06@ibsindia.org IMPORTANT DATES Last date for Abstract Submission : June 30, 2012 Notification of Acceptance : July 15, 2012 Last date for submission of full paper : August 30, 2012 Notification of acceptance : September 30, 2012

    Conference Registration Fee
    Nationality Category Before 15th Oct After 15th Oct
    Foreign Delegate Academician/ Industry $ 300 $ 350
    Research Scholar $ 150 $ 200
    Indian Delegate Industry Delegate Rs. 7500 Rs. 9000
    Academician Rs. 4500 Rs. 6000
    Research Scholar Rs. 3000 Rs 4500

    FOR MORE INFORMATION VISIT CONFERENCE WEBSITE: www.ibshyderabad.org/conference/ISDSI-IBS

    may 5, 2012 vol xlviI no 18

    PERSPECTIVES

    using consumption distribution as a poor substitute, we actually have income distribution data.

    What Ails the Statistical System?

    The answer to this question may lead us to the complex issues of governance and is likely to attract controversy. Nevertheless, there are certain drawbacks of the system which stand out. First, the subject of statistics is not receiving the attention and importance it deserves in the hands of public administration. There are many departments of the government which do not appear to consider quality data as an essential need for policy. They are happy with half-baked statistics and they show no awareness of the drawbacks of this data. Even the issues of slackness in governance seem to be getting reflected in embarrassing statistical errors of a severe nature.

    Second, the root of the malaise lies in the loss of development perspectives after economic liberalisation began in the 1990s. The flow of data as it existed in the controlled regime collapsed after liberalisation, but the reformers showed no enthusiasm for substituting the earlier controlled arrangement by a method befitting the liberalised environment. Year after year there has been only a steady deterioration.

    Third, one of the key components of reform policy has been fi scal prudence and consolidation, and as a consequence the worst to be affected has been the social infrastructure. Statistics fell in this category and there have been steep reductions in staff strength in the government’s statistical establishments. In the mean time, the complexities in mobilising statistics in diverse fi elds have vastly increased and the training requirements have grown manifold.

    Fourth, there has arisen a more fundamental issue of the loss of pride in statistics as a profession, particularly in public administration and public institutions like the Reserve Bank of India (RBI). It is found that the government and the institutions find it difficult to attract sufficient numbers of candidates and fi ll staff positions created after the regular attrition. Therefore, it is not a question of want of resources alone; it is also a question of the statistics profession being made more attractive intellectually.

    Fifth, the drawbacks in the statistical system as well as the unattractive nature of the profession in general, have also arisen because there is a serious fl aw in the organisational set-up of the statistical system. The NSC was to be a statutory body with permanent members and with executive and fi nancial autonomy, but the government chose to appoint an NSC with part-time chairman and members and did not make the body a statutory one. As an initial effort, this was no doubt commendable, but no effort has since been made to place the NSC on a statutory and permanent footing. The purity and reliability of statistics can be ensured only when the system is governed by an enduring institutional structure and is conferred independence and autonomy. Due to the absence of such autonomy, we see the sorry spectacle of very many statistical units being treated as appendages to the ministries and departments of the Government of India. There is a strong case for making the statistical units in different departments autonomous entities that primarily report to the NSC. Also, the NSC has a major role of guiding, and collaborating with, the state statistical bureaux. The absence of a statutory commission has truly hindered improvements of statistics in the states based on the recommendations contained in the Rangarajan Commission Report (2001). In this respect, the two new schemes, namely, the IMF-supported India Statistical Strengthening Project (ISSP) and the Thirteenth Finance Commission grant of Rs 1 crore per district for strengthening the district statistical system, should go a long way in effecting an improvement, but this is also conditional upon the central system being able to provide training and other guidance to the state bodies to absorb the resources so provided.

    Finally, India had acquired a commendable reputation for its sample surveys, but of late the survey units in the system have suffered a serious blow from a paucity of staff and the consequential dependence on part-time fi eld personnel. With a multiplicity of fi eld surveys expected of the National Sample Survey Offi ce (NSSO), the facilities for training of staff are found to be grossly inadequate. A recent World Bank report, “Modernisation of India’s Statistical System”, had placed the additional staff requirements for the systems at central and state levels at about 6,000. The introduction of information technology may have reduced that requirement, but the fact remains that the staff size, its quality and its training requirements have all been affected by a defective staff policy in the government. Various research papers have brought out how the sample sizes have been drastically reduced in diverse field surveys because of the inadequacy of the organisational strength at the NSSO or at the CSO level. As a result, both sample and nonsamples errors are feared to be high. Even statistical units in individual departments are deprived of adequate staff strength. Institutions like the RBI, which were involved in such surveys in the past, have taken a back seat and hence the required sample sizes have dwindled in relevant surveys.

    The drawbacks of the system are attributable to both the neglect by the government and the academic community. There are very few studies focusing on data adequacy and quality in India. There is yet another flaw arising from the neglect, essentially by the academic community. This is an absence of social audit of the data put out by public agencies claiming success in public programmes. Even various professional organisations do not, except for some rare exceptions, address database problems in their seminar programmes. Therefore, a stage has come for the intellectual community to come together and form an India Database Forum (IDF), on the lines of the International Data Forum.4

    The data issues raised at the international level are, of course, different from those we face in India, as enumerated above. The International Data Forum “aims to facilitate and coordinate international production and sharing of data for research in the social sciences”. However, one of the guiding principles set out for that forum, namely, to “act as a catalytic and visionary organisation to identify gaps in research data needs and to

    Economic & Political Weekly

    EPW
    may 5, 2012 vol xlviI no 18

    PERSPECTIVES

    promote a research-driven data agenda” (www.internationaldataforum.org), establishes some similarity with the objectives propounded in this note for the IDF.

    What Will the IDF Do?

    The tasks of the proposed IDF will be twofold; promoting research and advocacy. It must be made clear at the outset that the intention here is not to create an institution for producing data on any sector of the economy. That is the exclusive job of the government. Against the backdrop of data issues posed above, concerted actions are called for on two fronts. First, the IDF should promote research on the Indian database issues. Though these issues have gained importance in a variety of ways, they are not being addressed by professional bodies at all, or if they are addressed, they are done as tailend subjects.

    Second, there is the need, and ample scope, for advocacy work in matters relating to database issues. The statistical system is receiving stepmotherly treatment at the hands of the public administration. While the statistical cadre working in the government as well as in the statistical system in particular have a strong case for better attention from the government, they themselves cannot take up such issues. They cannot, for instance, deal with the problem of strengthening the statistical system, including the conferment of statutory status on NSC and making it an autonomous and independent institution. Likewise, there is a strong case for making statistical units in different departments and units autonomous and primarily reporting to the NSC. India is one of the few countries possessing no data on income distribution. Academicians and international organisations readily accept household consumption expenditure distribution as a surrogate for income distribution. Demands for undertaking such nation-wide integrated surveys on income, expenditure and savings have to be done in a concerted manner, and IDF can canvass for it in a systematic way. Likewise, there is the need for promoting research on green gross domestic product and generally on environmental issues.

    In the same vein, another important flaw in all programmes intended for the informal sectors and the poor concerns the absence of regular social audit of the data put out by public agencies claiming success in public programmes. Very often the nature and quality of data published are also inadequate to make a critical assessment of the social programmes in question. Demands for such independent social audits of public programmes and the authenticity of statistics on the programmes can be made in research articles or otherwise by the IDF.

    To correct the present malady, the IDF should galvanise the academic community to take interest in this subject, hold regular seminars, invite papers for workshops and conferences and produce a journal focusing on varied database issues. The universities and academic institutions should be goaded to introduce credit courses on the Indian statistical system and database issues at the MA/ MPhil and doctorate courses.

    Thus, the five key functions envisaged for the IDF are: (i) encouraging and promoting research on the extent and quality of statistics in different areas; (ii) activism for strengthening the statistical system including of the NSC and canvassing genuine autonomy for this body;

    (iii) seeking better focus on distributional issues in statistics gathering and dissemination while making public information and data policies on inclusive growth as well as on the scheme of financial inclusion; (iv) Demanding regular social audit of the data put out by public agencies, particularly in regard to claims of success in public programmes; and (v) encouraging universities to introduce credit courses on different types of data for their postgraduate, MPhil and PhD programmes.

    Notes

    1 Such concerns attracted the attention of the World Bank, which chose to fund a project called “Modernisation of the Statistical System in India” during 1998 but, as it was too ambitious requiring “high cost” and massive manpower with an additional staff of 6,000, the project was dropped in its original form. Beginning much earlier in 1972, the Indian Econometrics Society had held seven seminars on the Data Base of the Indian Economy during 1972 to 1997.

    2 One of the most studied and extensive reviews was that of S M Vidwans (2002a, b, c), himself a member of the National Statistical Commission, in three serialised articles in EPW. He then wrote: “About the system itself, in the beginning of the 1980s, I had noted the start of its deterioration and pointed out to the top central government statisticians where it was heading. Even then, when reality progressively unfolded before me during the commission’s work, I was unprepared for the sorry state of the system and the weakening of its institutions, the superficial perception of the managers of the system of the maladies afflicting the system and, more worryingly, the inappropriate, and perhaps harmful, prescriptions they intended to administer to it” (2002a: 3819).

    3 The earlier two studies of 1970 and 1980 were small-scale efforts by smaller committees (Ministry of Finance 1970 and Department of Statistics 1980). The series of efforts made by individual scholars associated with professional associations like the Indian Econometrics Society and the Indian Association for Research in National Income and Wealth (IARNIW) are sporadic, piecemeal attempts.

    4 I profusely thank R B Barman for providing me with a copy of the background paper on the International Data Forum (2008) (A Proposal Prepared by the Founding Committee, June 2008; Revised November 2008). Barman was a member of the Founding Committee of that Forum.

    References

    International Data Forum (2008): A Proposal to Establish a New Structure to Promote Knowledge about, Access to and the Development of Data for Social Scientific Research (Prepared by the Founding Committee, June 2008: Revised November 2008), www.internationaldataforum.org

    Srinivasan, T N (1994): “Data Base for Development Analysis: An Overview”, Journal of Development Economics, Elsevier, Vol 44(1), pp 3-27, June.

    – (2003): “India’s Statistical System: Critiquing the Report of the National Statistical Commission”, Economic & Political Weekly, Vol XXXVIII, No 4, 25 January.

    Vidwans, S M (2002a): “Indian Statistical System at the Crossroads I: Ominous Clouds of Centralisation”, Economic & Political Weekly, Vol 37, Nos 37 and 39, 14 September.

  • (2002b): “Indian Statistical System at the Crossroads II: Expansion of National Sample Survey”, Economic & Political Weekly, Vol 37, No 38, 21 September.
  • (2002c): “Indian Statistical System at the Crossroads III: Modernisation Project – Centralisation Par Excellence!”, Economic & Political Weekly, Vol 37, No 39, 28 September.
  • EPW Index

    An author-title index for EPW has been prepared for the years from 1968 to 2010. The PDFs of the Index have been uploaded, year-wise, on the EPW web site. Visitors can download the Index for all the years from the site. (The Index for a few years is yet to be prepared and will be uploaded when ready.)

    EPW would like to acknowledge the help of the staff of the library of the Indira Gandhi Institute for Development Research, Mumbai, in preparing the index under a project supported by the RD Tata Trust.

    may 5, 2012 vol xlviI no 18

    To read the full text Login

    Get instant access

    New 3 Month Subscription
    to Digital Archives at

    ₹826for India

    $50for overseas users

    Comments

    (-) Hide

    EPW looks forward to your comments. Please note that comments are moderated as per our comments policy. They may take some time to appear. A comment, if suitable, may be selected for publication in the Letters pages of EPW.

    Back to Top