Re: [Discuss-sudbury-model] scientific backing of Sudbury

From: Mike Weimann <mike.weimann_at_gmx.net>
Date: Tue Mar 14 06:56:00 2006

Hello,

I had offered to send Daniel Greenberg's text. No problem, here it is.

Mike

---------copy from: IDEClistserve_at_yahoogroups.com ----------

My name is Daniel Greenberg. I am one of the authors of Sudbury Valley
School Press’s book, “The Pursuit of Happiness”. As a courtesy, Mimsy
forwarded to me the posts of Mike Weimann, whom I had the pleasure of
meeting some years ago in the home of Mimsy and Mike Sadofsky.
Unfortunately, Mike’s posts do not advance the purposes of the
IDEClistserve, which I take to be centered on issues surrounding
democratic education - in particular, providing a forum for a discussion
of those issues based on factual and accurate information, and not on
fantasies and unsupported attacks. For the sake of the integrity of the
discussion on this list, I am asking Mimsy to forward to it my detailed
reactions to Mike posts. I will discuss each point separately.

(1) Mike starts by saying of the study underlying the book that it
“actually had almost no scientific/academic method underlying” and is
“of no convincing value for any academic research.” On what basis does
he say this? I have a considerable academic background, and am no
stranger to scientific studies. What does Mike actually know about the
method underlying this study, that leads him to make such an accusation?
  And what does he know about the “scientific/academic method” of
studying cohorts, that leads him to say this? Surely he owes the
members of this list a detailed explanation of such a dismissal of a
massive study that took over three years to prepare, and the efforts of
a large number of people, including people well versed in handling large
quantities of data, which this study certainly produced. The book
discusses methodology in some detail, but Mike obviously has not studied
this, nor does he seem to have the background to make such judgments
(more about this later).

(2) Mike goes on to state, “E.g. they had only 10 percent of the alumni
in their interviews (around 100 of a total number of 1000 roughly).”
This one sentence, given as an example (the only actual one) of the
“unscientific” nature of the study, reveals his total ignorance of the
clearly stated purpose of the study, which is discussed at length in the
book. The study did not set out to find out what happened to every one
of the over 1000 children who somehow passed through the school since
its founding in 1968. If it had done so, it would indeed have had no
value, since of those 1000+ children, there were students who came for
varying lengths of time (from a few months to fifteen years), came and
left at various ages (for a variety of reasons, none of which related to
the stated purpose of the study), and were widely varied in age at the
time of the study (from 4 years old to over 50 years old). Any valid
study must have a stated goal, and a well defined cohort that serves the
purposes of yielding legitimate results.
  The goal of our study was this: to show that people spending a
significant part of their school years at Sudbury Valley School, and
leaving to go to adult life (either directly into the world of work, or
into further advanced study at a post-high school level), did not suffer
adverse life outcomes as a result of the time they spent at Sudbury
Valley. In other words, to deal with the often-heard objection that
children allowed to be in a Sudbury Valley environment would have
limited options for their careers, and would feel discouraged and
disappointed in their life outcomes, this study set out to see whether
such an objection was valid.
  To attain this goal meaningfully, parameters had to be set as to what
we meant by “spending a significant part of their school years at
Sudbury Valley.” Our earlier exhaustive study, published in the book
“Legacy of Trust” (available from the Sudbury Valley School Press - a
book which, like “Pursuit of Happiness”, can be ordered online from the
school’s website, www.sudval.org), cast a wide net, including in it all
students who left at an age when they could go out into the world, and
who spent anywhere from one year upwards in the school. We excluded
from that study, and from the newer study, students who left the school
for a variety of reasons when they were younger, since any future life
outcomes would be influenced by their later school experiences after
Sudbury Valley.
  For the study in “Pursuit,” we narrowed the cohort to students who had
been at the school at least three years. This was a matter of judgment,
and our conclusions are limited to that cohort; but it seemed reasonable
to assume that students who at least spent the last three years of their
school experience at Sudbury Valley would be subject to sufficient
influence by the school to warrant being studied.
  We also required that the students be out of school for at least five
years, since it did not seem reasonable to study longitudinal outcomes
for students who had been away from Sudbury Valley for a shorter period.
  These parameters yielded a total cohort of 199 former students, of
whom 119 were successfully reached. We were not able to locate the
other 80, though we made considerable efforts to do so.
  As anyone versed in such studies will tell you, obtaining some 60% of
a potential cohort in the actual sample is extraordinary, and yields
results whose level of confidence is far higher than the level of
confidence warranted by most studies.

(3) Mike writes, “most likely the responses to the voluntary inquiry
came from positively impressed people, which biases the result
decisively (decisive factor).” It is outrageous to base a critique on
an imagined statement beginning with the words “most likely”. This is
nothing less than an explicit accusation of bias in the sample. In
fact, the only determining factor for inclusion in the study was
reachability, and many students were reached who had varied reactions to
their experience at the school. An accusation of bias needs facts for
its basis, not a vague “most likely” based on fantasy.

(4) Mike says, “From my memory I think there were very often unanswered
parts of the catalogue of questions.” I have no idea what he is talking
about. No one avoided answering any of the questions. Some questions
were not relevant to some people (e.g., what college did you attend
could not get a reply from someone who did not attend college), but
otherwise the interviews were complete. Mike’s memory - which he should
not have relied on when writing such a scathing critique - is wrong.

(5) Mike says, “I would have loved to get to know about some unsolved
questions.” Another vague statement. What unsolved questions? What is
he talking about?

(6) He writes, “I tried to figure out in several diagrams what part of
the basic population was selected - more than once without success.”
Perhaps this is another of his statements “from my memory”. In fact,
the tables and graphs are all carefully labeled, identified, and
specified as to the part of the cohort being represented. Not one of
the many, many readers who took the trouble to study the book carefully
complained about unspecified cohorts - precisely because there were no
such instances.

(7) Mike then adds another figment of his imagination: “I guess many
traditional school principals could write an almost similar story of
successes. Or at least is convinced he could.” He “guesses”? First of
all, the book is not a “story of successes.” It documents a host of
failures, disappointments, difficulties, etc., all of which it lays out
in detail, and without apology. Obviously, Mike does not remember
those, but they are there nonetheless. In fact, however, there does not
exist a single case of such a study, with such scope, size of cohort,
and detail of documentation, for any school, traditional, alternative,
democratic or otherwise. I wish there were “many traditional school
principals” who had the will, the money, the expertise, and the
personnel to perform such a study. We would all be better off for it.
One would expect that people interested in democratic education, or in
education in general, would be delighted that we broke the ice by having
the perseverance and determination to conduct our study, and forge a
path for others to follow and to improve upon.

(8) Almost as an afterthought, Mike adds, “And finally, a good,
professional evaluation should always be done by independent research,
not by the protagonists, in this case the founders of the school.” In
fact, as is clearly stated in the book (which Mike clearly also does not
remember), the study was conducted by people who were at arms length to
the school, and not associated with it in any capacity, as was the data
analysis. Another accusation based on fantasy.

(9) In his second posting, Mike adds (supposedly by way of
clarification), “An example of an unsolved point is for me the question
if the model would work in a different environment.” That, of course,
is obviously not an “unsolved point” of this study, but rather a
question for a different study altogether. It is as if someone
criticized a study of the relationship between smoking and cancer by
saying that it did not solve the question of the relationship between
sugar consumption and diabetes. What does the question Mike raises have
to do with the stated purpose of our study? Why does the fact that we
don’t address his question reflect badly on our study, which didn’t even
ask it?

(10) Then he goes on to add his description of Sudbury Valley’s former
students in the study as the kind of people who “have the economical
independence from their job (like many of the population interviewed by
you)”. Where on earth did he get that from? I can’t think of a single
person in the study who was economically independent of their job. Is
this from memory as well?

(11) Mike then goes on a little excursus about what a good study would
be like, exhibiting his complete lack of understanding of methodology in
such studies. He says, “my point since years is, if it is possible to
investigate/study people like rats . . . it is the scientific,
sociological approach. . . . If it doesn’t work for people, then my
question is - why not?” Well, that is the kind of question that is
addressed very early when one actually studies methodology, and Mike
should undertake such a study before asking such bizarre questions. It
should be obvious, however, even without much deep thought, that studies
with animals depend on carefully manipulated control groups that are
identical in all their basic characteristics to the group being
subjected to the particular study; whereas one does not have the ability
to identify sets of humans that are identical in their basic
characteristics (we don’t even know, in our current state of knowledge,
what those would be), nor does one have the ability to take one group of
humans and manipulate them through years of controlled living, all the
while observing how they behave by comparison with the other set of
humans being studied in their environment. The whole notion is absurd.

People wishing to understand the longitudinal study done by Sudbury
Valley should read it carefully, and judge it for what it is - for what
it states as its goals, for the actual (not imagined) methodology, and
for the validity of the very limited and specific conclusions it
reaches. No purpose is served, certainly not to the readers of this
list, by leveling a set of ill-conceived accusations and criticisms that
have little or nothing to do with what the actual study is.

It is not my intention to engage in a debate with Mike Weimann. I have
written this simply to set the record straight for those who value
integrity over unsubstantiated opinion

Mimsy Sadofsky
mimsys_at_comcast.net

Dannyasher_at_aol.com schrieb:
> It is a remarkable example of the narrow-minded prejudice with which Mike
> Weimann, supposedly a proponent of "scientific study", addresses this list.
> What person who was devoted to a fair and complete debate on an issue would
> expose this list to his extensive rejection of Daniel Greenberg's criticism from
> another maillist, without at least including the complete posting he is
> supposedly refuting that was made by Daniel Greenberg on that list? This is a
> fair sample of Weimann's objectivity.
>
> If someone is really interested in what was, in fact, Weimann's irrelevant
> and poorly conceived diatribe against Sudbury Valley's two extensive
> longitudinal studies of its former students, "Legacy of Trust" and "Pursuit of
> Happiness" (both available from the Sudbury Valley School Press, directly or through
> the school's website), I will be glad to forward his opening assault, and my
> long response, which the current post of Weimann's again attacks.
>
> Daniel Greenberg
>

-- 
Mike Weimann, Winsstraße 4, 10405 Berlin
http://kraetzae.de | http://idec2005.org
Received on Tue Mar 14 2006 - 06:55:22 EST

This archive was generated by hypermail 2.2.0 : Mon Jun 04 2007 - 00:03:14 EDT