Inaccuracies in Fable of the Fable
Although Mr. Brooks brings up some reasonable points, many of his points are either off base
or are just plainly wrong. This note examines his claims, using his headers in the order found on
his web page http://dvorak.mwbrooks.com/dissent.html. Our article to which he refers is
“Fable of the Keys” published in the Journal of Law and Economics and available on
http://ssrn.com/abstract=1069950 .
Dvorak is a Fraud:
We never claimed that Dvorak was a fraud or that the Navy study was a fraud. We did say that
the Navy study did not read like a scientific study, that it appears to have been overseen by Dr.
Dvorak with the possible bias that might bring, and that it contained flawed analysis.
Brooks says:
The [Liebowitz and Margolis] article dwells upon claimed Dvorak benefits that
seem outrageous, like the Navy's discovery (in the 1940's) that the cost of
switching a group of typists to Dvorak could be recovered in 10.3 days…But the
article either misses or avoids facts that make the favorable claims seem less
outrageous. The "10.3 day" claim doubtless considers that the Navy was already
training its QWERTY typists (of whom many were novices) periodically to boost
their speed. There was probably little additional cost to train in Dvorak instead of
QWERTY, so in the Navy's case the cost might easily be recovered.
In spite of Mr. Brooks use of the term “doubtless”, the Navy study claim that costs could be
recovered in 10.3 days did not, in fact, assume that the costs of training was zero because
training was going to take place anyway. The “Postscript” on his page even makes that clear
since the cost of the typists time in the retraining class is the largest single cost. He misses our
main point on this subject, however. The claimed full return of investment in 10 days implies an
amazing rate of return for any firm that might adopt it. From our footnote 14 [all of our quotes
are taken from our 1990 paper in the Journal of Law and Economics]:
If [the high rate of return were] true, this would be quite remarkable. A
converted Sholes [QWERTY] typist will be typing so much faster that whatever
the training cost, it is repaid every ten days. Counting only working days this
would imply that the investment in retraining repays itself approximately
twenty-three times in a year. Does this seem even remotely possible? Do firms
typically ignore investments with returns in the range of 2200 percent?
Brooks says:
The article also makes much of the fact that the Navy study and some others
used questionable practices that might favor Dvorak unfairly. But the article
cannot and does not say that these practices, such as averaging certain test
scores or throwing out "unfair" preliminary tests, did favor Dvorak, only that
they might favor Dvorak.
Not really. The changes clearly hurt the QWERTY scores. And they would almost certainly have
hurt the Dvorak scores if the same adjustments had been made to the Dvorak scores as long as
typists were improving with training. But these adjustments were not made to the Dvorak
scores. The net result was to hurt QWERTY. From our footnote 25:
It is not an innocuous change. We are told that three Qwerty typists initially
scored zero on the typing test but that their scores rose to twenty-nine, thirteen,
and sixteen within four days (at 20). We are also told that several other typists
had similar improvements in the first four days. These improvements are
dismissed as mere testing effects that the researchers wish to eliminate. But the
researchers made no effort to eliminate the analogous testing effect for the
Dvorak typists. Truncating the measurements to the average of the first four
days reduces the reported speed increases for the three typists with zero initial
speed by at least thirteen, twelve, and fourteen. Assuming the existence of two
other typists with similar size testing effects, removing this testing effect would
reduce the reported speed improvements by 3.6 words per minute lowering the
gain from 46 percent to 28 percent.
Brooks says
The "Fable" article puts a lot of stock in the total hours of Dvorak training
required to regain one's old QWERTY speed, doubting that the Navy Dvorak
group could really retrain in about 50 hours when, for instance, Strong's Dvorak
group took about 100.
There was a reason that we questioned the Navy Results as being unusual. Our paper explained
that the Navy study appears to be an outlier whereas the Strong results (regarding number of
hours required to catch up to old typing speed) were found by others. Here is the text from our
paper:
He [Yamada] cites a 1973 study based on six typists at Western Electric where
after 104 hours of training on DSK, typists were 2.6 percent faster than they had
been on Qwerty. Similarly Yamada reports that in a 1978 study at Oregon State
University after 100 hours of training typists were up to 97.6 percent of their old
Qwerty speed. Both of these retraining times are similar to those reported by
Strong and not to those in the Navy study.
Mr. Brooks presents a claim that Dr. Strong was biased against Dvorak.
The article [Liebowitz and Margolis] puts a lot of effort into defaming Dvorak,
(giving Strong as a source in one place) but it casually discards (in a footnote) the
possibility that Dr. Strong's GSA study--the article's star witness, and possibly the
worst setback Dvorak ever had--was itself subject to bias. Again, the article
either misses or avoids key facts that cast doubt on Strong's study.
Here is what we said in footnote 29.
Interestingly, Yamada accuses Strong of being biased against the Dvorak
keyboard (at page 188). He also impugns Strong’s character. He accuses Strong
of refusing to provide other (unnamed) researchers with his data. He also implies
that Strong stole money from Dvorak because in 1941, when Strong was a
supporter of Dvorak's keyboard, he supposedly accepted payment from Dvorak
to conduct a study of the DSK keyboard without ever reporting his results to him.
We think it provides a fair commentary on the dispute and doesn’t “discard” the claims
although we find Strong’s report to appear to be more scientific and convincing than the Navy
study and that might be what Mr. Brooks is objecting to.
Mercenary Huckster section:
Not much here. Our claim about how much money Dvorak received from the Carnegie
foundation ($130,000) comes from Yamada, a supporter of Dvorak who wrote an academic
article on the subject. Brooks reports a figure from Randy Cassingham ($14,000), who doesn’t
tell us where his quotes or figures are coming from. We never called Dr. Dvorak a mercenary
huckster, nor was that our intent. We do believe he was an avid fan of his work and that this
may have colored his view of his results. We also suspect that Lieutenant Commander Dvorak’s
superiors may not have been paying much attention to his tests, which is why the Navy was
confused about it when asked by reporters. according to the introduction to the Navy Study the
study was requested by Lieutenant Rice, a rank below that of then Lieutenant Commander
Dvorak, which is consistent with a view that Dvorak’s superiors might not have known what was
taking place.
Dvorak's Keyboard Failed, So It Must Be No Good/Market Economics Prove Dvorak Is Inferior
Mr. Brooks devotes two sections to this claim. But we never said anything like this. We were
examining evidence for whether inferior standards can dominate markets. The closest we might
have come to the flavor of Mr. Brooks claim was to point out that, given the 2200% claimed
return on investment, large firms that might undertake to train their typing pool with Dvorak
would have done very well if they adopted it and it was strange that none did, except for the
possibility that they didn’t believe Dvorak was better.
It Was Always Easy To Convert Typewriters
Brooks says:
In the '40s, the War Department estimated the cost at $25 per machine
(Cassingham). That was a lot of money in those days;
Our $5 figure for the cost of converting a machine came from Arthur Foulke’s book “Mr.
Typewriter” as we state in our article. The Navy study says that they had one brand of their
machines “thoroughly” inspected, overhauled and converted from QWERTY to Dvorak for $15.
Some of that cost was presumably the thorough overhaul so that the cost was less than $15.
The point being that the cost of the typist was significantly greater than the cost of the machine
and that the cost of converting to the Dvorak design shouldn’t have stood in the way of its
adoption if the Navy study numbers were correct. Whether it is $5 or $15 or even $25 is largely
irrelevant. A (slow) typist in the Navy cost $1752 per year according to the Navy Study.
QWERTY Was Designed Ergonomically
Brooks argues that there was no reason for Sholes to create an ergonomic keyboard. But it
doesn’t really matter if he tried to create an ergonomic keyboard or not. Intentions are less
important than results—what counts is whether the keyboard is any good or not. We examined
studies reported in the ergonomics literature about different keyboard designs (a point Mr.
Brooks never discusses). Those studies found little difference between QWERTY and Dvorak.
We quoted one of those studies in our paper:
The Dvorak keyboard does a good job on these variables, especially A and B: 67%
of the typing is done on the home row and the left-right hand balance is 47-53%.
Although the Sholes [qwerty] keyboard fails at conditions A and B (most typing is
done on the top row and the balance between the two hands is 57% and 43%),
the policy to put successively typed keys as far apart as possible favors factor C,
thus leading to relatively rapid typing.
If QWERTY is a good keyboard it doesn’t matter if this was intentional on Shole’s part or not.
That is perhaps of historical interest, but is irrelevant to the current discussion. We also talk
about QWERTY winning typing competitions, which it did, and winning in market competition,
which it did. The evidence indicates that it is a good keyboard design.
Postscript
I am not sure who is writing the email found in Brooks’ postscript, but I will attribute it to Mr.
Brooks, since he posted it. It mainly talks about the Navy Study and the differences in
measurement that were applied to the QWERTY and Dvorak typists.
Mr. Brooks says:
Before you buy into the Margolis and Liebowitz slant on the Navy study, consider
this: The "start score" for the Navy's Dvorak group was actually their QWERTY
"endscore" after an average of 419 hours of training! If the Navy had averaged
the Dvorak group's starting scores, they would have had to do it on QWERTY
before switching keyboards. (Otherwise Dvorak would have posted _truly_
incredible gains.) Averaging four days' QWERTY typing would have made no
significant difference to the baseline because the Dvorak group were already
experienced QWERTY typists.
The claim of 419 hours training is an estimate of lifetime training, fairly irrelevant for the
purposes of the Navy Study, and quite misleading. The Navy Study’s estimate of previous
training was taken as the number of months in a typing class, whether in high school or
business school, multiplied by 20 hours. Many of the typists in the group had been out of high
school for 3 or 4 years (one as long as 8 years) but their high school (or business school)
training was counted when the Navy Study determined the efficacy of the former QWERTY
training compared to the new Dvorak training. Obviously, examining the typing speed three
years after training ended is hardly the proper way to measure the efficacy of the training and
one of the reasons not to take the Navy Study seriously, since it seems like such a set-up. The
later part of the Navy Study, examining the retraining of QWERTY typists on QWERTY was a
“supplementary report” that looks like it was performed because the original examination
comparing very old and depreciated training in high school to current training on Dvorak was
clearly bogus.
The final statement in the above quotation, that averaging the first four days of typing speed
would have no impact because these were experienced typists is also incorrect. Both the group
learning Dvorak and the group retraining on QWERTY were ‘experienced’ typists. They just were
not very good typists, on average well below the Navy’s deemed level of competence. For
example, the 3 slowest typists in the 14 person group learning the Dvorak keyboard had
starting typing speeds of 14, 16 and 25 words per minute. The average typing speed for the
group was 33 words per minute whereas Navy competence was 50 words per minute. With
such low initial scores, there might have been important initial improvement during those first
4 days such that taking the average of the first 4 days might decrease the speed increase
considerably. That is exactly what happened with those retraining on QWERTY where the
average initial speed for 3 typists was 14 words per minute or less and taking the average of the
first 4 days instead of the initial score reduced the gain from 48% to 26% for the entire group, a
very substantial change. That fact that this wasn’t done for the typists switching to Dvorak is
another reason to doubt the veracity of the results.
Mr. Brooks continues:
As for averaging the QWERTY group's last four test scores, M&L don't mention
that the Navy compensated for the averaging by computing the gains based on
_four days less_ than the actual training period.
Mr. Brooks either does not understand the arithmetic or the testing process, or both. The
average retraining of QWERTY typists was 158 hours. By reducing the training for 4 days (@ 2
hours per day; although the report doesn’t say how many hours of training occurred per day,
Mr. Brooks assures us that it wouldn’t be more than 2 hours since anything more is a waste of
time), you get a reduction of 8 hours out of 158, or about 5%. But the same adjustment
reduced the increase in typing speed by 40% (from 46% to 28%, see footnote 25 reproduced
above) and we do not know by how much the averaging at the end reduced the measured
typing speed (assuming speed was still increasing). A 5% improvement (decrease in the
denominator) does not compensate for a 40% decrease (in the numerator).
Mr. Brooks continues:
Given that, the only way averaging could hurt the QWERTY group's net gain is if
their scores were _decreasing_ at the end of the study.
Mr. Brooks gets it backwards. If scores were increasing at the end, then the average of the last
4 days is lower than the final score so using the average lowers the measured increase.
It seems Margolis and Liebowitz did a very poor job of reading the Navy study.
For another example, they howl at the Navy's claim that the cost of retraining
was amortized 10 days after retraining. They can't even read their own writing:
in one place M&L say 10 days after the end of training; in another they say 10
days after the _start_ of training (their emphasis).
In our Journal of Law and Economics article we mention the 10 day recoupment period twice.
The first time it is this: “including David who refers, without citation, to experiments done by
the U.S. Navy that had shown that the increased efficiency obtained with the DSK would
amortize the cost of retraining a group of typists within ten days of their subsequent full-time
employment.” Later we say “Compare this to the claim David makes about the Navy study's
results that the full retraining costs were recovered in ten days.” We cannot find in our Journal
of Law and Economics article the statement Mr. Brooks attributes to us.