On the Uses and Abuses of Metrics for Life

ITZ: metrics creators warning people about the dangers of misapplying their inventions and promptly getting ignored.

Intelligence Quotient (IQ)

tl;dr: attempts to separate nature vs nurture + binet basically saying that intelligence can't be measured

Binet did seek “to separate natural intelligence and instruction” (1905, p. 42) in his scale [...]

Yet, beyond this obvious desire to remove the superficial effects of clearly acquired knowledge, Binet declined to define and speculate upon the meaning of the score he assigned to each child. Intelligence, Binet proclaimed, is too complex to capture with a single number. This number, later called IQ, is only a rough, empirical guide constructed for a limited, practical purpose:

The scale, properly speaking, does not permit the measure of the intelligence, because intellectual qualities are not superposable, and therefore cannot be measured as linear surfaces are measured (1905, p. 40).

Moreover, the number is only an average of many performances, not an entity unto itself. Intelligence, Binet reminds us, is not a single, scalable thing like height. “We feel it necessary to insist on this fact,” Binet (1911) cautions, “because later, for the sake of simplicity of statement, we will speak of a child of 8 years having the intelligence of a child of 7 or 9 years; these expressions, if accepted arbitrarily, may give place to illusions.” Binet was too good a theoretician to fall into the logical error that John Stuart Mill had identified—“to believe that whatever received a name must be an entity or being, having an independent existence of its own.”

mismeasure-of-manch. 5

tl;dr: anticipation of misuse of IQ as justification to weed out troublesome kids, potential of IQ to set off a self-fulfilling prophecy

Binet also had a social motive for his reticence. He greatly feared that his practical device, if reified as an entity, could be perverted and used as an indelible label, rather than as a guide for identifying children who needed help. He worried that schoolmasters with “exaggerated zeal” might use IQ as a convenient excuse: “They seem to reason in the following way: ‘Here is an excellent opportunity for getting rid of all the children who trouble us,’ and without the true critical spirit, they designate all who are unruly, or disinterested in the school” (1905, p. 169). But he feared even more what has since been called the “self-fulfilling prophesy.” A rigid label may set a teacher’s attitude and eventually divert a child’s behavior into a predicted path:

It is really too easy to discover signs of backwardness in an individual when one is forewarned. This would be to operate as the graphologists did who, when Dreyfus was believed to be guilty, discovered in his handwriting signs of a traitor or a spy” (1905, p. 170).

mismeasure-of-manch. 5

tl;dr: IQ invented for identifying "special needs" children, not for measuring intelligence

Not only did Binet decline to label IQ as inborn intelligence; he also refused to regard it as a general device for ranking all pupils according to mental worth. He devised his scale only for the limited purpose of his commission by the ministry of education: as a practical guide for identifying children whose poor performance indicated a need for special education—those who we would today call learning disabled or mildly retarded. Binet wrote (1908, p. 263): “We are of the opinion that the most valuable use of our scale will not be its application to the normal pupils, but rather to those of inferior grades of intelligence.” As to the causes of poor performance, Binet refused to speculate. His tests, in any case, could not decide (1905, p. 37):

Our purpose is to be able to measure the intellectual capacity of a child who is brought to us in order to know whether he is normal or retarded. We should therefore study his condition at the time and that only. We have nothing to do either with his past history or with his future; consequently, we shall neglect his etiology, and we shall make no attempt to distinguish between acquired and congenital idiocy.… As to that which concerns his future, we shall exercise the same abstinence; we do not attempt to establish or prepare a prognosis, and we leave unanswered the question of whether this retardation is curable, or even improvable. We shall limit ourselves to ascertaining the truth in regard to his present mental state.

mismeasure-of-manch. 5

Gross Domestic Product (GDP)

From a section titled "USES AND ABUSES OF NATIONAL INCOME MEASUREMENTS", in this report to Congress from the creator of the GDP measure:

tl;dr: warnings about misplaced concreteness of quantitative measurements

The valuable capacity of the human mind to simplify a complex situation in a compact characterization becomes dangerous when not controlled in terms of definitely stated criteria. With quantitative measurements especially, the definiteness of the result suggests, often misleadingly, a precision and simplicity in the outlines of the object measured. Measurements of national income are subject to this type of illusion and resulting abuse, especially since they deal with matters that are the center of conflict of opposing social groups where the effectiveness of an argument is often contingent upon oversimplification.

nat-inc-29-32ch. 5

tl;dr: warnings about paying attention to assumptions, inadequacies for measuring shadow work, inequalities, net productivity

These constitute highly valuable uses of national income measurements, but only if the results are interpreted wHh a full realization of the definition of national income assumed, either explicitly or implicitly, by the measurement. Thus, the estimates submitted in the present study define income in such a way as to cover primarily only efforts whose results appear on the market place of our economy. A student of social affairs who is interested in the total productivity of the nation, including those efforts which, like housewives' services, do not appear on the market, can therefore use our measures only with some qualifications. Secondly, the present study's measures of national income, like all such studies, estimates the value of commodities and direct services at their market price. But market valuation of commodities and especially of direct services depends upon the personal distribution of income within the nation. Thus in a nation with a rich upper class, the personal services to the rich are likely to be valued at a much higher level than the very same services in another nation, characterized by a more equitable personal distribution of income. A student of social affairs who conceives of a nation's end-product as undistorted by the existing distribution of income, you'd again have to qualify and change our estimates, possibly in a marked fashion. Thirdly, the present study's estimate of national income produced is based in part, like most existing estimates, upon the prevalent legal and accounting distinction between gross and net income of business enterprises. To a student of social affairs whose concept of net productivity does not agree with the prevailing practices of separating net from gross income, especially by corporations, our estimates will obviously present a somewhat distorted picture of the nation's net product.

nat-inc-29-32ch. 6

tl;dr: original creator of GDP warning that national welfare != income measures e.g. GDP

All these qualifications upon estimates of national income as an index of productivity are just as important when income measurements are interpreted from the point of view of economic welfare. But in the latter case additional difficulties will be suggested to anyone who wants to penetrate below the surface of total figures and market values. Economic welfare cannot be adequately measured unless the personal distribution of income is known. And no income measurement undertakes to estimate the reverse side of income, that is, the intensity and unpleasantness of effort going into the earning of income. The welfare of a nation can, therefore, scarcely be inferred from a measurement of national income as defined above.

(emphasis added)

nat-inc-29-32ch. 6–7

tl;dr: warnings, once again (unheeded), about the importance of paying attention to definitions, methods, and assumptions employed in deriving income estimates, i.e. paying mind to the "metaphysics" à la Schumacher

The abuses of national income estimates arise largely from a failure to take into account the precise definition of income and the methods of its evaluation which the estimator assumes in arriving at his final figures. Notions of productivity or welfare as understood by the user of the estimates are often read by him into the income measurement, regardless of the assumptions made by the income estimator in arriving at the figures. As a result we find all too commonly such inferences that a decline of 30 percent in the national income (in terms of "constant" dollars) means a 30 percent decline in the total productivity of the nation, and a corresponding decline in its welfare. Or that a nation whose total income is twice the size of the national income of another country is twice "as well off", can sustain payments abroad twice as large or can carry a debt burden double in size. Such statements can obviously be true only when qualified by a host of "ifs."

nat-inc-29-32ch. 6–7


nat-inc-29-32 Foreign, Division of Economic Research of the Bureau of, and Inc. of New York City. Domestic Commerce in close cooperation with the National Bureau of Economic Research. 1934. “National Income, 1929-1932: Letter from the Acting Secretary of Commerce Transmitting in Response to Senate Resolution No. 220 (72nd Cong.) a Report on National Income, 1929-32.”. 73rd Cong., 2d Sess. Senate. Doc. 124. U.S. Govt. Print Off. ↩︎ 1 2 3 4

mismeasure-of-man Gould, Stephen Jay. The Mismeasure of Man. ↩︎ 1 2 3