What does TOEFL test sometimes by uploaddoc


									What does TOEFL test sometimes?
Vyara Istratkova & Ellie Boyadzhieva
South-Western University of Blagoevgrad

General Introduction
        The choice of this topic has been provoked by some observations upon the
preparation work and the successful accomplishment of some tasks by would-be
Bulgarian takers of the TOEFL.
        Traditionally, Bulgarian takers of the TOEFL have very good results. It is
mainly teenagers in their last year at high schools (most notably English Language
schools) that take the test, the most common purpose being to continue their education
at American universities. Many of them go in for courses offered for preparation for
the TOEFL examination.
        The Bulgarian candidates‟ approach to the test is based mainly on the
presumption that their high level of language proficiency is the necessary prerequisite
for success. It does hold true to some extent as it is necessary indeed. Yet, language
proficiency is not a sufficient condition for success, and that is what we will try to
prove below.

Aims of the TOEFL
        The TOEFL in used as a standard measure of the „English proficiency‟ of the
candidates taking it. The TOEFL score is a compulsory requirement of the admission
offices of different educational institutions and organizations so that foreign
candidates are considered for admission. (In the 1996 edition on p.6 we read: “Many
Universities use TOEFL scores to fulfill the foreign language requirement for doctoral
candidates whose first language is not English” (TOEFL 1996:6). Actually, this
applies to nearly all (American) universities and it refers to undergraduates as well. An
interesting point is that the TOELF is to be taken even by British students.

The mate rial in focus and the authors’ goal
        As it is well known the TOEFL consists of three sections: Listening
Comprehension, Structure and Written expression and Reading Comprehension &
Vocabulary section and it tests the candidates‟ five skills - listening, structure, written
expression, vocabulary and reading (TOEFL 1989:8).
The material in focus presents samples from the 1st , 2nd and the 3rd part of the
Listening Section of two different editions of TOEFL which (also) differ in their
format – the Listening Section of the Sixth edition of Barron‟s TOEFL (1989) consists
of a statement-restatement part, conversations and mini-talks, whereas in the Eight
edition of Barron‟s TOEFL (1996) the Listening Section is comprised of short
conversations, longer conversations and talks. The examples included (in the corpus)
are illustrative, the authors‟ goal being to show that sometimes something more than
language proficiency and proficiency in how to approach the examination is needed
for the successful accomplishment of the listening tasks of the test.
Theoretical background
               Tests are measurement instruments and (according to Carrol, 68) a
               “procedure designed to elicit certain behavior from which one can make
               inferences about certain characteristics of an individual” (Carrol 1968:46).
               In fact, tests are (or, ideally, should be) simulated real- life situations. On
               the basis of the performance of the candidate one should be able to make
               predictions about his/her language behavior in real life. The test itself
               should be based on the real needs and serve as basis for predictions about
               the ability of the students to cope with people and matters in reality.
        In the theory of testing and evaluation, it is a well-known fact that it is “neither
theoretically nor practically possible to define either an absolutely perfect level of
actual language performance or an individual with perfect language ability” (Bachman
1990:11). This can be regarded as a direct consequence of the fundamental dilemma in
foreign language testing, namely that “the tools we use to observe language ability are
themselves manifestation of language ability” (Bachman 1990: 9).

         Our personal observations show that in the TOEFL examination the most
troublesome area appears to be the Listening Section of the test. Why should reading
for example be easier than listening? After all, both are receptive skills. Yet, the
phenomenon observed is not a surprising one - listening comprehension is found to be
among the most difficult tasks for the learners due to several reasons. First, spoken
language is ephemeral, and the examinees cannot refer to it whenever they want to as
is the case with Reading comprehension texts. Consequently, they have to rely mainly
on their memory abilities, which differ throughout individually. Second, examinees
are pressed for time as the time is limited within the length of every particular text.
Third, they have to develop different strategies for the different types of tasks
depending on the types of exercises.
         Accordingly, candidates sometimes fail to successfully accomplish the
listening tasks. This may be owing to the fact that they do not know the meaning of a
word or an expression. However, even if they did, it still might not make sense in the
context, as the ideas of the text (the discourse) present a different point of view on the
reality. When so, situations occur in which the students make wrong decisions even in
cases when they know every single word in the string because of the fact that what
they have heard merely does not meet their expectations, and sounds weird to them.
Thus, candidates often fail to see the point because the language in use is strongly
culturally biased.

       We shall now consider several types of most problematic discourses for
Bulgarian takers of the TOEFL excerpted from the Listening sections of the two
TOEFL textbooks already mentioned and classified as academic and political
discourse, education, different kinds of services, everyday practices, and social

    I. Academic environment
    The examples connected with academic environment outnumber the rest. In
        general, the
main factors for making mistakes in items reflecting the specific American academic
environment are a consequence of the crucial differences between the American
educational system and the one we have in Bulgaria. Basically, these differences
concern the syllabus structure, the types of courses offered, the existence/lack of credit
system, the academic positions. Not rarely do students face the problem of the so
called false friends, as it is the case with the word “professor” for instance.

e.g. 1. (Barron‟s TOEFL, 1989)
         A: I‟d like to take Dr. Sullivan‟s section of Physics 100, but my advisor is
teaching it too, and I don‟t want her to be offended.
         B: Who cares?

The question is: What should the woman do?
The correct answer is A: The woman should not consider her advisor in the decision.

e.g. 2 (Barron‟s TOEFL 1996)
          A: So the course is closed. This is terrible. I have to have it to graduate.
          B: You are o.k. Just Dr. Collin‟s section is closed. There is another section that
is still open, but nobody knows who‟s teaching it. It‟s marked staff.

The question is: What would the student probably do?
The correct answer is C: Enroll in the section marked „staff‟
(cf. B. Graduate at a later date; D. Find out who‟s teaching the other section -

        In both examples, in order to work out the right answer Bulgarian test-takers
should know what the duties of the advisor are and what his/her functions are
(advisors are members of the academic staff at American Universities). Moreover, at
American universities one and the same subject is taught by two (sometimes even
more) people and the students are free to choose whose course to attend which
presents different practice from what we have in Bulgaria. One is also to be aware of
the existence of credit system at US universities.

e.g. 3 (Barron‟s TOEFL 1996)
        A longer conversation between a student and a professor from which it
becomes clear that the student couldn‟t register for the class of the instructor because
the course was closed by the time he got to the front of the line. The two problematic
questions are: 1. What‟s Mike‟s problem (The correct answer: He must have the
permission of the instructor); 2. What does the professor decide to do(T the correct
answer: Allow Mike to take the class this term).
       Here again, different regulations at US universities are reflected which do not
have equivalents within the Bulgarian educational system - Bulgarian students are to
be aware of the fact that there is a competition among the students to register for a
course on first-come first-serve basis as well as of the possibility for exceptions
allowed by the professors themselves.

e.g. 4 (Barron‟s TOEFL, 1996)
A talk concerning assessment and evaluation:
         “You have a midterm examination the last week of October, and a final
examination the second week of December. The midterm is worth 25 points and the
final is worth 50 points. That leaves 25 points for the project that …… and you have
several choices to fulfill that requirement. You either write a paper or make a half
hour presentation”……

The question is: What are the course requirements?
The correct answer is D: A midterm, a project and a final exam
(cf. A. A midterm and a final exam; B. A midterm and either a final exam or a project;
C. A midterm and a paper or a presentation)

        Obviously, the language in use is again culturally biased - knowledge is
required about the content of the notions midterm and project (written paper/oral
presentation); projects do not constitute part of the continuous assessment, they are
part of the final grade. Students themselves own up that the whole seems to be a
complete mess and they take pot luck when answering the question.

   II. Education

e.g. 5 (Barron‟s TOFEL, 1996)
        The text presents a talk – in fact, this is a public service announcement telling
about possibilities offered by a college for distance learning provided by the help of
video telecourses. The question is : What is the announcement mainly about? (The
correct answer: Video telecourses). In the explanatory notes the other two options
(The Sun up semester programme & The community college campus) are said to be
„secondary‟ as they have been „used to develop the main topic‟.

        Bulgarian takers of the TOEFL exam are generally not aware of the existence
of distance education as these are usually young people at school age.This type of
education seems to be quite extended in the US whereas in Bulgaria such a possibility
has been provided for only a couple of years now by few institutions and does not
have as long traditions as in the US.

       III. Political discourse
e.g. 6 (Barron‟s TOEFL 1996)
This is a lecture on history with the following key expressions:
        Each political party… nominates a slate of electors pledged to support the
party‟s nominees… each state has the same number of electors in the college as it has
members of Congress …. registered voters go to the polls to choose the electors. The
ballots list only the names of the candidates… This vote by the people for electors is
called the popular vote and the candidates who receive the most popular votes win all
the electoral votes in a state.

        The text includes a lot of specific terms and the test-takers need specific
subject knowledge about the political electoral procedure. At short notice and after
having heard the talk once only they have to identify all the names and to adjust them
to the roles played by the different groups of people at the elections. The very word
college is rather confusing - “An organized body of persons with shared functions and
privileges” (Oxford Reference Dictionary). Thus, it is beyond doubt that language
proficiency is far from sufficient if a Bulgarian student is to answer the questions:
How are the people nominated for the electoral college? What is the popular vote?

   IV. Everyday practices – food and drinks

e.g. 7 (Barron‟s TOEFL 1996)
        This example presents a longer conversation between two speakers in front of
a coffee machine which has just stolen the man‟s money. Finally they decide to go to
the library to try the vending machine there. The questions to be answered are: 1. Why
did they decide to go to the library? (The correct answer:To have coffee); 2. What
prompted the conversation? (The answer: The speaker wanted coffee); 3. What do the
speakers mainly discuss? (The correct answer: The coffee).

         Vending machines are typical for all public places in the US and other
countries, yet hardly ever seen in our country. So, you can go to any public place
(library including) and be sure to find a machine there whereas to a Bulgarian a library
is a place to only borrow or read books. An additional problem is the fact that the word
“vending” has no equivalent in Bulgarian – no word for non-existing artifact.

e.g. 8 (Barron‟s TOEFL 1996)
        A: What do you want on that?
        B: Everything and extra catsup, too, please.
The question is: What does the man mean?
The correct answer is: lettuce, pickles, onions, mustard, mayonnaise, and catsup

     The TOEFL textbook explains: “Everything is an idiomatic (?) expression that
     means all
the condiments included”. Obviously, this item does not test language proficiency at
all. You have to have lived in the US for some time to know the expression. As
Bulgarian takers of the TOEFL examination are usually young people at school age,
most would have no prediction as to what the phrase means.
   V. Health services

e.g. 9 (Barron‟s TOEFL 1989)
        This is a conversation between a pharmacist and a patient of Dr. Williams‟s,
the doctor being supposed to give a prescription.
        Key phrases: Aspirin is the strongest medication I can give without a
prescription; I can call him…. Dr. Williams will give me a pain prescription over the
        The question to be answered: 1.Where does the conversation take place? (The
correct answer: In the drug store); 2. Why did the man call Dr. Williams? (The
correct answer: To get a prescription for the woman on the phone)

         Something one should take into consideration here is the existence of different
words in the two varieties - drugstore in American and chemist‟s in British English. It
is true though it does not usually cause problems for Bulgarians, but it might well be a
problem for some native speakers of British English.
         The main problem for a Bulgarian here might be the behavior of the chemist
which seems rather strange since it is not possible to get a prescription on the phone in
our country. What is more, in Bulgaria, no prescription would be needed a for a pain

   VI. Services

e.g. 9 (Barron‟s TOEFL 1996)
        A: You turn on the TV by pulling on this button. The heat control is on the wall.
Will there be anything else, Ma‟am?
        B: No, thank you.
The question is: What can be inferred about the man?
                     A. He‟s a TV repairman
                     B. He‟s a bell boy (the correct answer)
                     C. He‟s a tailor.
                     D. He‟s a security guard.

        In order to decide on which the correct answer is, one is to first identify the
settings - deduction should be made about a place with both a TV and a heating device
for which someone is responsible. This is more or less a matter of common sense,
however only possible for experienced travelers since very few hotels provide bell
boys services in Bulgaria and these are usually very expensive ones.

   VII. Social behavior

e.g. 10 (Barron‟s TOEFL 1989)
This is a restatement example:
        My invitation has RSVP printed at the bottom.
                     A. I should dress formally.
                    B. I should tell the hostess whether I‟ll go (the correct answer)
                    C. I should take liquor.
                    D. I should buy a ticket.

       It is of crucial importance here that the abbreviation be interpreted correctly.
The abbreviation itself is a borrowing from French and means repondez s‟il vous plait.
This phrase has almost no circulation in Bulgaria except on very formal occasions .
We should not except the average Bulgarian to know it.

Some conclusions
   1. The TOEFL textbooks do never provide any information about cultural
      specifics and concentrate on grammar and lexis only. Consequently, many
      prospective test takers go in for courses which prepare them for the successful
      sitting of the examination.
   2. The courses but most often provide instruction only in grammatical and
      vocabulary issues and focus mainly on the techniques and building strategies to
      complete the test tasks within the time required.
   3. The material discussed above and the situation observed lead inevitably to
      some important consequences concerning the type of instruction necessary for
      the preparation for the TOEFL not only in BG but elsewhere. The basic
      presumption is that every candidate targets higher scores, hence a correct
      answer to every test item seems to be crucial for him/her.
   4. This puts additional load onto the instructors as they have to provide at least
      the minimum information on several cultural specifics, and especially in the
      academic discourse as this topic area presents abundance of different cultural
      practices included in the test content.
   5. From everything said so far one general question follows: who will make a
      better instructor for the TOEFL examination – a native or a non-native teacher?
      In the authors‟ opinion a non-native teacher seems to be better suited to
      perform this task as cultural issues involve implicit or explicit comparisons
      with another culture and only a non-native teacher can possess the knowledge
      and intuition what the similarities and the differences in the target culture are
      so that to elicit the most problematic topics and issues.
   6. In the Bulgarian environment the above problem might sometimes have an
      alternative solution, namely – a British native-speaker teaching in TOEFL
      courses. Having in mind that sharing one mother tongue does not necessarily
      imply sharing one and the same culture, we could conclude that a British
      teacher is often a better choice than an American one because the former will
      also feel the specifics of the social practices adopted in the US.
   7. The analysis of the material shows that the TOEFL ( as well as many other
      standardized proficiency tests of English – cf. FCE) tests language proficiency
      as well as cultural awareness although it is often pointed out that culture should
      be subject to a different testing procedure. Still, in our opinion it is often
      absolutely impossible in cases of proficiency tests to divide language from
      cultural competence as it is intrinsically embedded into the use of the particular
8. One additional point to be made here is that the preparatory TOEFL courses
   can successfully provide information about the social, educational, behavioral
   and political specifics in the US .
9. Finally, we can conclude that the TOEFL aims at testing not only the language
   competence of non-native speakers of American English, but also their
   communicative competence which encompasses both linguistic competence
   and cultural awareness without which language cannot be used appropriately in
   different social contexts. So, the TOEFL reflects the general tendency in ELT
   which is to move further on from pure accuracy (or language competence) to
   pragmatic competence.

To top