Journal
Myth Busting & Metric Making: Refashioning the Discourse about Development in the Integral Community
November 01, 2008 23:56

Zachary Stein
Developmental Testing Service
Harvard University Graduate School of Education

Last month I presented a couple papers at the first Biannual Integral Theory Conference. They were well received. However, as much as I flapped my lips to whoever would listen, I felt that I returned to the Northeast with a great deal left unsaid. Human development is one of the key foci in the discourse we are building. But over the years I've come to see a real need for the refashioning of this focal point. Roughly speaking, we are not as developed as we should be in our thinking about development. While I did what I could to remedy this by flapping my lips out in California, I've decided to start writing things down. With the help of the editors at Integral Review and Integral Leadership Review I'm working on a set of articles that will allow me to get some things off my chest. What I offer here is a kind of preamble to that project, which will unfold over the next 9 months or so.

But what do I mean when I say we are not as developed as we should be in our thinking about development?

One of the papers I presented at the ITC addressed the development of reasoning about key topics in AQAL. That is, I offered a map of how thinking unfolds in this knowledge domain, e.g. levels in thinking about the quadrants, levels in thinking about lines, and, to the point, levels in thinking about levels. A version of this paper will eventually be out in The Journal of Integral Theory and Practice (JITP). But until then, and to make clear what I'm talking about, here is a rough and speculative vertical rational reconstruction of levels in the development of thinking about levels. It’s important that this story be held lightly. I don't have time or space to issue all the caveats I need to (see Stein and Heikkinen, 2008 in JITP for background on the methods behind this; or go to www.devtestservice.com).

The Development of Reasoning About Levels of Development

If we look at college-educated adults, the first level is abstract mappings on our metric (roughly Orange in Wilber's colors). At this level, developmental levels are treated like simple stereotypes. Whole persons are classed as being at a level, which is typically understood in terms of a single developmental model (e.g. Spiral Dynamics). Development is understood as a kind of simple "growth to goodness", with ignorance at the bottom, science in the middle, and spirituality at the top. Particular levels gain more attention than others and function as more or less entrenched stereotypes, expressing preferences that are not necessarily developmental (e.g. "you are so green").

The next level is abstract systems (roughly Green in Wilber's colors). At this level, reasoning about levels involves giving some primacy to the construct of altitude, which frames and organizes a variety of developmental models. Persons are understood in terms of their relative development in various lines, which are identified with different developmental models and theorists. The concept of a center of gravity supplements this differentiated view and justifies whole person assessments. The relation between levels and other aspects of Integral Theory becomes explicit; the relation between states and levels complicates the simple notion that spirituality is "at the top." Generally, there are elaborate ideas about how developmental levels are implicated in all kinds of issues (politics, religion, ecology, etc.).

Then there is reasoning at single principles (roughly Teal in Wilber's colors). At this level, reasoning about levels involves explicit ideas about the limits and affordances of different developmental methods and models, which are framed in terms of arguments about the conditions enabling their valid use (i.e. scoring systems, interview procedures, etc.). The idea of "growth to goodness" is problematized both by concerns over issues of horizontal health and intra-personal variability, and by concerns about the accuracy of different assessment methods. These complexities of method and application temper and complicate speculation on how developmental levels are implicated in a broad range of global problems.
 
The top of what we can accurately measure is principled mappings (roughly Turquoise in Wilber's colors). At this level, reasoning about levels involves the adoption of a post-metaphysical stance toward the task of evaluating people. The provisional, bounded, and multi-perspectival nature of all models and methods is admitted, and a set of meta-theoretical principles guides a recursive process of continually refining developmental models and methods in terms of both theory and practice. A broad and explicit philosophical discourse comes to supplement evaluative discussions concerning the notion of "growth to goodness," as the human potentials that characterize the highest levels and the future of civilization are seen as collective constructions for which we are responsible.
 
Now, the way I see development (i.e. Fischer's Neo-Piagetian perspective) suggests that we roam up and down these levels all the time, depending on context and support, etc. No one is at a level; we inhabit levels only for certain periods of time and in certain company. Moreover, you may be more developed in your reasoning about the quadrants than you are in your reasoning about levels, or more developed in you reasoning about important interpersonal issues then you are in your reasoning about Integral Theory (again, see Stein and Heikkinen, 2008 in JITP). Generally, where you are is not my concern (in part because you are all over the place).

To speak loosely, my concerns are about our collective center of gravity. What's the level of the general discourse about development? From where I sit, it appears that we need to move from systems to principles (i.e. make the so called "leap to Second Tier") as a community in the way we hold our ideas about development. I see this transformation in terms of two myths that should be prominently and permanently busted by the Integral Community.
 
Myth Busting and Metric Making
The myth of the given has been named and is already generally disparaged. The other myth remains nameless but plagues our efforts. The myth of the metals is introduced here as the second myth requiring critical attention. Importantly, if we choose to jettison both myths we must refashion the practice of developmental assessment and the discourse surrounding it. The first myth raises epistemological issues and its critique should lead us to pay more attention to how our developmental metrics are made. The second raises social, ethical, and political issues and its critique should lead us to pay more attention to how our developmental metrics are used.

The myth of the given: Wilber has aptly brought attention to the myth of the given, which plagued epistemologies until it was lampooned by Kant and Co. during the post-metaphysical turn. This is a myth about what we know and how we know it. Jettisoning this myth is an act of epistemic humility wherein we admit that the world appears only in light of the constructs we employ to question it. Knowledge is not simply given via sundry experiences (that's the myth); knowledge is made via experiences that are systematically disclosed. Therefore concerns about how we make knowledge take center stage.

Despite its being busted by many (e.g., Peirce, Baldwin, Piaget, Goodman, Sellars, etc.), this myth has persisted in many disciplines, including developmental psychology. Rich descriptions and explanations of how development unfolds are produced and consumed (especially in the Integral Community) with very little attention to the methods, models, and metrics that make them possible. We love the knowledge, but could care less about how it was made. So the myth of the given rears its head.

What if were we to jettison this myth and adopt a properly post-metaphysical approach to developmental assessment and theory? Things would be different. For example, the issues surrounding the making and refining of developmental metrics (those things which disclose the data) would take primacy over the presentation of various stage models and narratives. We would turn away from the stories describing development and towards the making of the metrics that justify these stories. This kind of intra-disciplinary self-reflection represents a level of methodological sophistication that characterizes many sciences and is especially important for those aiming to generate useable knowledge. The truth is that as a discipline we have not attained this level of sophistication and, at this point, there is no generally accepted endeavor aimed at exercising quality control vis-à-vis the proliferation of developmental approaches.

The FDA is good example of a quality control agency that mediates between researchers and the lifeworld by bringing attention to the methods and claims being made in the laboratory. If we were to thoroughly dislodge the myth of the given in developmental studies and choose to adopt an epistemologically responsible approach to building and using developmental metrics we could learn from agencies like the FDA. Not any theory and its concomitant interventions should reach the market. We need to be concerned first about what justifies the approach and then about how exciting or revolutionary it is.  Right now we are mainly concerned with the latter. How many peer-reviewed empirical studies are published confirming the efficacy and validly of Spiral Dynamics; not stories about its use or antidotes of success, but studies on the validity of it basic constructs and measures? Answer: next to none. But how popular is this approach? Answer: It is one of the most popular approaches out there and its language is ubiquitous. But this in not just about Spiral Dynamics, comparable criticisms could be leveled at other approaches. In general, we need to refashion the discourse about which developmental approaches are best.
 
The myth of the metals: The second myth is Plato's. In the Republic one finds the myth of the metals. This is a myth devised by philosophical social engineers to justify the structure of a society that is organized hierarchically according to capabilities and dispositions. The myth goes like this: some citizens have blood mixed with Gold, others Sliver, Iron, or Copper. This metallic endowment defines a person's essence and allows their being ranked and assigned a role in society. Assessments of traits and capabilities serve this differential distribution: Gold and Silver are indicative of Leaders and Warriors, Iron and Copper of Merchants and Farmers.

Plato is concerned with justice and believes that the contingencies of human nature make it necessary to engineer a harmonious society. He envisions a complex and radical public educational system, with various forms of psychological and physiological assessment for evaluating individuals and putting them in their place. The myth of the metals is essential to this task. The idea is that if we plan to use assessments of capabilities in the structuring of society then we must create ideologies to justify the differential distribution of opportunities that result from those assessments. Such an ideology is to be devised and disseminated by the leaders as a way of making sense of and enforcing a caste system to those incapable of grasping the abstract ideal of justice, which is its true justification. The myth of the metals is way of framing the use of psychological assessments. It suggests that such assessments are capable of defining the essence of a person and determining the range of what is possible and preferable for them.
     
Now, it was Karl Popper who first made absolutely clear the totalitarian strain running through Plato's political vision. And the myth of the metals is the lynchpin because it masks coercive social engineering practices by disguising them as the fateful and acceptable decrees of authorities. Have no doubt, the ideal of justice as a harmonious system of rights and responsibilities stands, despite the flaws of any particular vision regarding its institutionalization. But today we rightly balk at the notion of handing someone his or her identity and role in society based on a set of assessments administered by a small group of experts. Yet, something like this, something like the myth of the metals, seems to be evident in many cultures that employ (or are looking to employ) developmental assessments.
 
Many who consider themselves developmentalists think it is valid to make holistic developmental assessments that determine the essence of a person. Moreover, because higher-levels are hypertrophied it is thought that more developed people are better people. Generally, it is thought that we use developmental assessments to find out how good people are so that we can then give them the acclaim, trust, and responsibility they deserve. Even a cursory familiarity with the current discourse surrounding the practice of developmental assessment reveals that it looks like a lot like the myth of the metals.
 
Busting this myth means refashioning the discourse surrounding the use of developmental assessments. This has at least two implications. On the one hand we must tease apart the objective and descriptive from the evaluative and prescriptive. Simple growth-to-goodness models overlook the radically non-obvious evaluative import of being assigned a level score. Since Kohlberg, the naturalistic fallacy has plagued developmental psychology. It appears that we are easily seduced into bootstrapping our languages of evaluation from the languages we use to objectively describe developmental patterns and pathways. Determining the value of being at a level is different from determining the objective fact that one is at a level. Facts and values are not the same, although the myth of the metals would have them be.
    
On the other hand, and here we tie back into the first myth, there is overwhelming evidence to suggest that our metrics are limited and that we can't touch the true complexity of human development. In this light, the idea that a holistic assessment could tell us about the essence of a person is absurd and flagrantly ideological. Developmental assessments, at their best, can only paint pictures of the differential distribution of capabilities within persons. We can't assess people as a whole, we can only assess their performances in particular domains in particular contexts. The myth of the metals would bestow on some group of experts a unique kind knowledge and insight that vastly outstrips the kind of knowledge gained through the responsible use of a developmental assessment system.

Moreover, we should administer developmental assessments in order to promote development, not just to rank people and assign them their position in an organization or social group. The myth of the metals would have us use assessments solely to administer the allocation human capabilities (and to inform us about the concomitant worth of one another). If we loose the myth then we must rethink this use. It seems vastly preferable to wed assessments to educative efforts at all levels and use assessment solely for the purpose of promoting development. There is an important ethical point here about the use we make of our evaluations of others, i.e. whether others are seen as ends-in-themselves, or as means to an end.

Developmental Assessment: What's Possible and What's Preferable
If we begin to openly disparage the myths discussed above we will need to refashion the practice of developmental assessment and the discourse surrounding it. Of course, when we loose one myth we often create another. The vision of a future informed by rigorous and accurate developmental assessments needs to be realistically articulated and re-articulated. No one who has been convinced of the importance of a developmental perspective can deny that major institutional reforms are needed in business, government, and education. Current global socio-cultural conditions are resulting in an increasingly uneven distribution of capabilities between people and an increasingly dangerous stratification of cultures according to the complexity of world-views. While these gaps sometimes fall along fiscal fault lines, their amelioration is better grasped in terms of the promotion of learning processes, not economic interventions. Moreover, those most privileged face the ever-increasing complexity of a global problem space. Individuals in all sectors in all roles face an unprecedented demand for conditions conducive to life long learning. Citizens confront the indelible pluralism and chaos of a fragmented public sphere in which numerous political problems are presented from multiple perspectives, making it seem that those conditions conducive to life long learning are in fact the conditions conducive to a democratic form of government. These are the kinds of big-picture problems best diagnosed in developmental terms.

But despite our enthusiasm as to its prospects, the science of developmental assessment is just now reaching maturity. James Mark Baldwin, who first glimpsed the possibility of a thoroughgoing developmental and integral psychology, was pioneering up through WWII. Piaget died in 1980. His students are still with us today. The aforementioned progenitors of developmental science were prodigious enough to leave room for various progeny. We face not one, but many, developmental psychologies. Time will tell the fate of each. Clearly, as discussed above, questions about their relative validity and utility bear on questions of their probable longevity. Of course, questions of validly and utility are, more often than not, questions of methodology. Reliable and accurate developmental assessments are hard to achieve. And we've only been working on them for about 50 years. So it should come as no surprise that the best methods for assessing developmental change in persons have yet to be invented. This kind of epistemic humility is scientific and its council is one issued in all applied sciences: work with the tools you have, but be on the look out for better ones.

We also need concerted philosophical, ethical, and political reflections on role and future of developmental assessments in our society. The question: "what should we want from developmental assessments?" is a good one. As are the questions: "what can we realistically expect from them now, in the future, and in principle?" Steering a trajectory forward requires a vision for the future of the discipline that grapples with both what is possible and what is preferable.

Zachary Stein
Developmental Testing Service
Harvard University Graduate School of Education

Part 2 of this series can be found here.

Vertline-top
Vertline-bottom



« recent entry | return to index | previous entry »


  © 2015 Ken Wilberhome | what's new | professional | personal | cultural | social | cool stuff site design by ursa minor