OUT OF THE BOX: AGAIN AND AGAIN

Roger Herriot at the Census Bureau

William P. Butz

U.S. Bureau of the Census

Upfront, I admit I am a fan of Roger Herriot. I admire who he was and what he did. And, by and large, how he did it. So this discussion is a personal one. I don't offer an exhaustive review of Herriot's life work, nor even a summary. I do offer these observations, though, as an analysis of how he accomplished so much. It is a critical analysis. As I say, I admire how he did it...by and large. I hope in addition to being critical, it is also productive and funny. For Roger was always productive and often funny, although I almost never heard him be personally critical of anyone.

Table 1 shows a list of items. It is noteworthy in three respects. First, it is a diverse list: scientific and technical, statistical and economic, organizational and programmatic.

Second is the importance of many of these items as "hot topics" on today's National policy agenda. This relevance is obvious in many cases. Some specialized knowledge is required to see it in others.

Third, Roger Herriot was instrumental in inventing, discovering or implementing, as the case may be, each item. Not only instrumental: I will develop the argument this morning that Herriot's involvement--No, let me strengthen my case a little--Herriot's engagement in a problem was a sufficient condition for the discovery of a solution and for the programmatic implementation of that solution. This is an extraordinary claim to make about anyone while perusing a list of his or her life activities. To claim it about a federal civil servant working always with and through coworkers and always with or against bureaucratic structures and rules, may be considered foolhardy. In this rare case, I think the statement is supportable. I'll repeat it: Roger Herriot's engagement in a problem was a sufficient condition for the discovery of a solution and for the programmatic implementation of that solution.

It is not useful to try to trace Herriot's contributions to these items by way of his publications. Although these run to four pages of citations, I find documentation of only a handful of his contribution in this way. Neither have I discovered how he accomplished so much by considering how Roger thought about a problem. By and large, this is a mystery to me as it was, I think, to him on the occasions we talked about it. James Gleick, in his recent biography of Richard Feynman, quotes Murray Gell-Mann on how Feynman solved problems. Gleick reports that a physicist studying with Gell-Mann at CalTech in the 1950's asked Gell-Mann whether Feynman's own problem-solving methods were the same as the methods Feynman proposed in unpublished lecture notes that were circulating. "Gell-Mann says no, Dick's methods are not the same as the methods used here. The student asks, well, what are Feynman's methods? Gell-Mann leans coyly against the blackboard and says, Dick's method is this. You write down the problem. You think very hard (Gell-Mann shuts his eyes and presses his knuckles parodically to his forehead.) Then you write down the answer." This seemed to be Herriot's general method as well. I don't learn much from it.

What, then, made Roger Herriot so immensely productive? What distinguished him from most everyone else? Certainly, he had a solid background in economics and statistics as well as rich organizational experience, having worked at the Census Bureau 22 years. And he was real smart. But some other people also have these attributes. I see the answer to the puzzle of how Herriot did it less in how he thought about problems than in his attitude toward problems and in his behavior with others in dealing with problems. Herein, I propose, is the key to his sufficiency. These are the attitudes and behaviors that, in my view, fundamentally set Herriot apart:

1. He had unflagging optimism that a solution could be found and an unwillingness--a categorical unwillingness in my experience--to admit that something could not be done, or to accept that assessment from others. Several instances:

• Consider the shambles of federal support for the Survey of Income and Program Participation that threatened hard to leave the new survey stillborn in 1982 after seven years of development. Roger remained optimistic that a package of design features, questionnaire content and early deliverables could be forcefully put on the table by the Census Bureau Director and bought by the other Cabinet departments, OMB and Congress. This did come to pass.
• Herriot was always optimistic that a new system of household income data could be built, based on the three legs of the Current Population Survey, The Survey of Income and Program Participation and IRS records, with CPS providing the timeliness, SIPP the detail and IRS records the accuracy for some items, all modeled together. This has not yet come to pass and shows little life at the moment, but, were he here, Roger would smile and say, "It'll happen" ...then sketch for us how the science, politics and budget will come together.
• One thing Roger and I disagreed about: whether data items not provided by a respondent in a survey but available on some administrative record for that same person should be substituted directly into the statistical record and subsequently analyzed and released. I thought not, Roger thought so, and he was optimistic that it would happen. I had to be constantly vigilant to keep him from getting it sneaked in somewhere.
• Scores of times I have heard Roger drawl out in meetings, "I probably don't know all the details and complications here, but I don't understand how that could take so long (or be so complicated). It seems like all you have to do is..." Boy, a lot of people didn't like to hear that!

2. Herriot was unwilling to choose between two good things when there appeared to be insufficient time or resources to do both, or when the two things seem technically inconsistent. Instead, he refused to consider either/or, and displayed a remarkable talent for finding a way to do the essential parts of both, or to do a third and different thing that accomplished the first two plus more. An example:

• Consider, for example, the continuing conflict between the expanded provision of survey microdata to users--especially survey data linked to related administrative records--on the one hand, and the protection of respondents' confidentiality and privacy, on the other hand. This conflict arose 40, maybe 50 specific times in Roger's career, but he refused to view it as a conflict! He proposed and implemented solution after solution in case after case--some solutions statistical, some technical, some bureaucratic, some legal. This reached its zenith at the National Center for Education Statistics where Herriot developed protocols of informed consent, data collection, data matching, record formatting and user contracts that, in my estimation, just barely crawled through the available space between all the constraints.

3. He continually searched for low-cost spinoffs from ongoing activities. He asked a whole lot of questions like, "Now that we have TIGER, what else can be done with it?" or "As long as we're doing these Statistical Briefs, why not give them to the field representatives so they and the survey respondents can know what's happening with the data?" or "If we're going to use SAS for data analysis, why can't we try it for generalized data processing?" And he formulated answers to such questions.

4. He didn't wait for a mandate to do something, or even for permission. "If you see a vacuum of power, expertise, accomplishment, go ahead and fill it." Roger had little patience for those who complain that they can't do something because they don't have the grade or the title or it isn't in their job description. "Don't ask permission," he drawled. "Do it. People will probably be pleased and if some aren't, you can apologize later." If you do need some authority you don't have, all that is required, Roger would remind me, is for the boss to support you twice in a row on a decision. I have found this a subtle but powerful technique for shifting power around in the bureaucracy.

5. He begged or stole the time of collaborators to work out solutions--usually along lines suggested by himself--and to get these solutions implemented in actual programs. Roger Herriot was legendary at getting people who were supposed to be working for someone else to work informally for him instead. Paula Schneider, who succeeded Roger as Chief of the Population Division, told me once, "Roger goes up and down the hall getting Pop Division people to work on his stuff. (But don't worry, I can handle it.)" Jay Waite, Chief of the Demographic Statistical Methods Division, mentioned to me about three months into his then new job, "It says here on the Division roster that Bob Fay is working for me, but as nearly as I can tell, he's really working for Herriot." Dan Weinberg, Chief of the Housing and Household Economic Statistics Division, replied to a request, "Jack McNeil, Enrique Lamas and Chuck Nelson can't do that. They're working on that idea of Herriot's ."

In these dealings, Herriot was the master of unstructured assignments. He would sit down with someone and give them a scrap of paper that made little sense. He would start talking about it. And big things would eventually result. The method of multiple synthetic imputation of occupations between the 1970 and 1980 census occupation classifications got its start this way.

Roger was a productive collaborator (some 20 different co-authors appear on his CV), but he was a truly great instigator, applying to this endeavor prodigious ingenuity and stealth. Two years ago several high officials at the National Center for Education Statistics, important customers of the Census Bureau, let us know that they weren't altogether pleased with the work we were producing for the money they were sending us. Much of the trouble, when together we figured it out, was, as I understood it, that some of our people were actually working hard on a project of Roger's, rather than on what they were being paid to do by his colleagues at the Department of Education!

6. He was totally unconcerned about who gets the credit, uncommonly modest. He liked to say of the Population Division staff: "My people make me look good." I must have heard this 30 times over the years. Roger Herriot was a personification of the adage, "There is no end to what can be accomplished if you don't care who gets the credit."

7. He always did some real work. Herriot feared losing track of the details. This was a very real concern, often expressed to me. He told me, "You have to know how the work is actually done before you can be effective in changing it." And on other occasions, "They can argue you down, if you don't know what they're actually doing."

By the summer of 1988, Roger had led his small team of co-designers to a fleshed-out description of the revolutionary Integrated System of Area Statistics, which would replace the census long form in 2000 or 2010. He had conspired with Bruce Johnson to use this model as the basis for a series of staff retreats to think about the 2000 census. With each retreat, new criticisms of Roger's plan emerged. The sample couldn't be controlled, the phone numbers couldn't be matched, the estimated coefficients of variation were biased. Several days later, a new paper would emerge. Even the name changed, to the Decade Census Program. Roger took Census Bureau alumni to lunch to try out his program. More revisions followed. Soon, critics and kibitzers couldn't keep up with the flow of paper. Their critiques became passé shortly after they were distributed. For example, in late September 1988, an excellent critique delivered the conclusions of one technical review committee. The critique states up front, "Our comments pertain to the August 16, 1988 draft. Later proposals, including the possible use of mail questionnaires, are not discussed." Multiple proposals in just 45 days! The author of this critique, Chip Alexander, became the principal designer of the Decade Census Program's current powerful incarnation, the Continuous Measurement Program.

Charles Darwin, in an 1871 letter to his son, wrote this, which reminds me a good deal of Roger Herriot:

"I have been speculating at night, what makes a [person] a discoverer of undiscovered things, and a most perplexing problem it is. Many [people] who are very clever--much cleverer than discoverers--never originate anything. As far as I can conjecture, the art consists in habitually searching for causes or meaning of everything which occurs. This implies sharp observation and requires as much knowledge as possible of the subject investigated."

Roger got this knowledge by keeping his hand in. There were always at least a couple of things that Roger knew more about than anyone else.

8. He worked a problem in his mind continuously--in the office, in meetings on other subjects, commuting, at home--until he had a solution, and had checked it out with a few other people. Herriot did use Feinman's method: "You write down the problem. Then you think very hard. Then you write down the answer." But that second stage could last days or sometimes weeks.

So there you are. Eight characteristics of Roger Herriot's attitude toward problems and of his behavior with others in dealing with problems that, in my view, account for his success in solving problems and implementing solutions.

His record wasn't perfect. Table 2 lists the items I can come up with that fully engaged his energy but did not come to pass. He liked to tell the stories about these few things that didn't work out. In these stories, the culprit was invariable himself. It was what he overlooked or presented to the wrong person that messed things up. I heard about these failures far more often than the many successes. He was apparently rolling them over in his mind... learning from them.

His methods weren't perfect either. Some of the eight characteristics I've discussed aren't always productive. Here are four others that certainly are not:

1. Roger was not an effective communicator. He had only average writing ability and was quite poor in front of a large group. He spoke softly and slowly, frequently haltingly. When he was Director of the Census Bureau, Jack Keane called me into his office one day and said, "Roger is outstanding in so many ways. Why don't you get him to take a workshop in public speaking so he can express himself better in front of a group?" I suggested to Roger that he do this, and he agreed. Every now and then Jack asked me if Roger had taken the course. Finally we gave up. I don't think Roger ever took it. If he did, it didn't do him much good.

In one important circumstance, though, Roger was quite an effective communicator. This was when an argument was developing, when voices rose and tempers frayed. At these times, Roger's voice grew even softer, the words coming even more slowly. He had a calming influence. And it was then, sometimes, that he would float the germ of an idea that might possibly satisfy both sides of the argument at once.

For other occasions, Roger knew his weakness and built up around him colleagues, notably Gordon Green, who were strong public speakers and press briefers.

2. He was not an effective administrator. He certainly wasn't good at budget monitoring, administrative reporting or meeting administrative deadlines. What's worse, I suspect he didn't care about these important matters, either, because too often he failed even to arrange for others to do these things and keep him in line.

Altogether, it can be said that Herriot disregarded and frequently disdained bureaucratic procedures. I got fair warning in my first week on the job. My boss was explaining how to write my performance plan for 1983. "Look at Tom Walsh's plan," he advised. "It's a good example of an excellent plan: the right number of elements and good specificity for each. For the other extreme, look at Roger Herriot's." A year later, the Chief of the Budget Division pleaded with me, "Can't you do something about Herriot?!" I got more than a few such requests.

Now it is true that, in those days, we had colleagues who seemed to believe that the core mission of the U.S. Census Bureau was to submit budget documents on time and in prescribed format; to control expenditures carefully; to undertake no hire, no capital procurement, no diversion of work time, unless the budget for these items was absolutely guaranteed to perpetuity. For these people, Roger Herriot did not contribute to the Agency's mission!

Even by legitimate and essential administrative criteria, though, Roger came up short. It hurt his effectiveness and it hurt him, within the organization.

3. His style and results didn't always please people. Certainly they didn't please many of our administrative colleagues. Moreover, they could displease technical staff who had worked long and hard on a solution to a problem and on a decision strategy for adopting the solution, only to hear Herriot drawl out some different--and better--solution off the cuff at a meeting late in the game. I well remember one irritated colleague following me to my office after a meeting. "Who the hell does he think he is," I heard, "coming in at the last minute with an idea like that after staff have been working on it for four months?!"

After World War II, General Eisenhower stated three principles he tried to follow in dealing with the enemy. First, never question his motives; in his own mind he thinks he's right. Second, never embarrass him in public; that will only make him fight harder. And third, never cut off his escape route. If you do, you must destroy him and he might destroy you. Give him a way out. It seems to me that Roger Herriot followed this prescription. He was a gentle guy. But some of his ideas were not gentle, and they could and did make their originator unpopular.

4. His methods don't scale up. Herriot was like a successful public school principal. He got results by bending rules and stealing the best staff. If everyone tried this, I don't think it would work. (We could certainly do with five or ten times as many trying it, however!)

A physicist colleague remarked about Richard Feynman, "There are lots of people who are too original for their own good, and had Feinman not been as smart as he was, I think he would have been too original for his own good." Likewise, Herriot was original and he was smart. It worked.

Roger Herriot's engagement in a problem was a sufficient condition for finding a solution and getting it implemented. This does not mean that those of us who worked with Roger on something weren't productive, even vital. It means only that if we hadn't been on that job, Roger would have found, somehow, a way around our absence. Herriot was sufficient in this sense, which, I submit, is quite a meaningful one in a bureaucracy. If, then, Herriot's engagement was sufficient, the challenge for those around him became: "How can I get Roger engaged in my problem?" or "How can I keep him from getting engaged in my problem?"--the latter when one already has a solution of one's own or a process of one's own to find a solution.

For many years, some of the most stimulating, the most productive, the most dangerous, the most fun moments at the Census Bureau came unexpectedly when Roger Herriot stood in your office doorway, without knocking, and drawled apologetically, "You gotta few minutes? I wanna show you something interesting."

TABLE 1

ROGER HERRIOT'S MAJOR INNOVATIONS

At the Census Bureau

• Model to simulate taxes in The Current Population Survey.
• Analytical and content aspects of the Survey of Income and Education.
• Methodology for updating income estimate for revenue sharing.
• 1973 CPS - IRS - SSA Exact Match Study.
• March CPS income supplement.
• Matching - based imputation system for growing CPS non-response.
• Income Survey Development Program (ISDP).
• Probability coding for IRS address match to improve population estimates.
• Multiple synthetic imputations method to bridge occupation classifications between 1970-80 censuses.
• Longitudinal definition of households.
• Non cash income estimates.
• Rescue and initiation of The Survey of Income and Program Participation.
• SIPP recurring report series.
• SIPP Executive Committee and decision structure.
• Statistical Briefs.
• After-tax income estimates.
• Publications Opportunity Committee.
• SIPP Longitudinal Research File.
• Decade Census Program (which became Continuous Measurement).
• Idea of a continuously maintained address list for surveys and census.
• SAS analysis of spells and transitions in SIPP.
• SAS as a processing system tool.
• State income estimates by combining CPS years.
• Decennial Access and Profiling System (DAPS-90).

At the National Center for Education Statistics

• Modernized statistical standards review procedure.
• Greatly expanded micro-data dissemination.
• User-friendly data base on the nation's school districts.
• Innovative electronic data dissemination.

TABLE 2

ROGER HERRIOT'S FAILURES

• Proposal for a Census Bureau Visiting Scholars Program.
• Matrix Sampling in the 1990 Census.
• Longitudinal Analysis of Mature Persons ("LAMP") - a long-term panel retirement and aging survey.

Modeling SIPP, CPS, and IRS data in a new system of income and poverty statistics.