VILTS: A Tale of Two Technologies

Marikka Elizabeth Rypa
Nuance Communications
Patti Price
SRI International

Abstract:
The Voice Interactive Training System (VILTS) is a language-training prototype developed to help improve comprehension and speaking skills. The system incorporates two related technologies: speech recognition and pronunciation scoring. Speech recognition allows students to navigate through units by using oral communication skills. Pronunciation scoring, validated through correlation with expert raters, provides assessment of speaking skills. We discuss the motivation for the program, the interdisciplinary efforts involved, and the resulting system architecture. We also describe challenges and trade-offs in designing activities using unscripted material and in integrating new speech technology. Finally, we discuss system evaluation and opportunities for future directions.

KEYWORDS

Voice Interactive Training System (VILTS), Natural Language Processing, Speech Recognition, Evaluation

INTRODUCTION

ECHOS, the French version of the VILTS, was developed to help improve listening and speaking skills by using state-of-the-art speech recognition technology.1 Designed to support language learning and maintenance of skill at beginning, intermediate, and advanced conversation levels, the VILTS lesson architecture stresses learner-centered navigation through listening and speaking activities. Two related technologies underlie the system design: speech recognition and pronunciation evaluation. In the following sections we motivate the program, describe the system architecture, explain the use of speech technology in the language pedagogy, and discuss future directions.

385

MOTIVATION AND GOALS

Active speaking skills are central to the needs of most language learners, and people learn best through contextual, culturally valid, interactive listening and speaking (e.g., Lee & VanPatten, 1995). In the last few years, great strides have been made in multimedia educational tools; features such as sophisticated animation, graphics, and audio input and output capabilities are increasingly used to develop engaging interfaces. Until recently, however, user interaction has been limited by the lack of robust speech recognition technology, and oral user input was largely confined to recording and playback. Recent advances in speech technology have yielded high-performance, continuous-speech, speaker-independent recognition. This technology supports the development of more sophisticated interactive language learning software and enables users to navigate using spoken utterances in active exchanges with the system. We explore several uses of speech recognition technology in interactive language learning. We also consider another speech technology, pronunciation scoring: the evaluation of nonnative speech as compared to native pronunciation calibrated through correlation with expert human raters. These two technologies, speech recognition and pronunciation scoring, place different requirements on the lesson interface; the motivation for the VILTS program was to bring together advances in each technology to support language education in away that was most technically feasible and pedagogically valid.

We have used technology to present a broad range of natural, authentic speech from a diverse set of talkers in an immersive, learner-centered pedagogical experience. As core material for lessons, we created a structured corpus balanced for age, sex, topic of discussion, and complexity of language use. These materials were incorporated into an engaging, interactive, flexible architecture to support learner-centered navigation through various levels, topics, and activity types in an environment that could foster individual learning styles. Our interdisciplinary collaboration linking speech technology with language pedagogy has resulted in a prototype system with interesting new possibilities for language learning. Our work has focused on French, as seen in ECHOS, although we have also explored the teaching of Spanish and English, and all our algorithms are portable to other languages. Inherent in an interdisciplinary effort are both benefits and challenges. The advantages include a broadening of ideas and input to the lesson activities, which contributes greatly to the richness and range of learning interactions developed for the project. The challenges include the incorporation of diverse viewpoints in a cohesive, engaging, pedagogically valid system that use sthe technology appropriately.

386

THE VILTS ARCHITECTURE

The modular VILTS architecture was designed to be extensible to other languages and to a wide range of activity types. Dialogues or other types of exchanges can easily be incorporated as core materials, and activity types are developed for existing and new languages. The VILTS can complement classroom instruction; it does not include a full language learning program, but it enables this extension. VILTS uses the speaker-independent, continuous speech recognizer from Nuance Communications, based on SRI's speech recognition technology The acoustic models used were custom-developed by SRI for language education. These models are wideband (as opposed to telephone bandwidth) and are based on a homogeneous pool of French speakers (Parisians) as well as English learners of French at various levels of experience. SRI developed the pronunciation scoring described in this paper.

A Conversational Core

The ECHOS version of the VILTS uses spontaneous, unscripted French conversations on various topics, supplemented by excerpts read from the French newspaper Le Monde. Conversations were collected on ten common topics, including travel, health, education, environment, and politics. Linguistic complexity of the conversations was controlled by the interviewer's questions. "Beginning" level conversations contain relatively simple vocabulary and constructions and were elicited by simple (usually "yes/no") questions. "Intermediate" and "advanced" conversations contain progressively longer speech segments and more complex and idiomatic expressions. These levels were elicited, respectively, through questions that are more complex and questions aimed at eliciting a monologue from the conversant (who claimed to be an expert in the topic of the interview). A pool of 100 native speakers in Paris recorded the conversations, representing a variety of speaker characteristics and speaking styles.

Instructional Flexibility and Learning Styles

The VILTS lesson architecture is shown in Figure 1. The content is conversational, thematically-based activities based on the topics of the French interviews we collected. Although the architecture suggests order implicitly in the layout of the interface, users can also navigate freely through levels, topics, and activity types. The left-to-right screen layout of activities, beginning with listening comprehension activities, encourages the student to proceed from listening to speaking, that is, from passive to

387

more active skills. The suggested path proceeds from comprehension practice through speaking to reading longer segments aloud. Students can, however, choose how to access the materials and activities based on individual interests and needs. The flexibility of the architecture accommodates different learner styles, from more structured, incremental learning to free exploration. For example, after system logon, students can move directly to a lesson unit or review units already completed. This flexibility was motivated by research in second language learning that shows the primacy of individual styles of learning (e.g., Oxford, 1995) and suggests that no single sequence of instruction or modalities is optimal for all learners.

Figure 1

The VILTS Lesson Architecture

0x01 graphic

Students may choose one of three levels (beginning, intermediate, or advanced) and then choose a topic that interests them from those available at that level (e.g., travel). Users can remain at one level longer to become more confident with the material, or they can forge ahead in a more exploratory style. Although the system structure is designed to foster initial practice of listening comprehension skills with an emphasis on speech and minimum exposure to text, users can choose activities that offer slower versions of the spoken samples as well as text support (e.g., transcriptions). This approach accommodates both oral- and text-based learning styles. The structure of the lesson activities is illustrated in Figure 1 above. Each of the five activity types increases in complexity from top to bottom and consists of several instances with different content based on

388

the conversations or the related newspaper text. The architecture also allows the student to engage in pronunciation exercises based on areas of diagnosed weakness.

Because our audience is adult learners who are assumed to be capable of self-monitoring, we have provided a learning environment in which users make decisions and control the sequence of instruction (with reference, of course, to feedback from the system). This architecture contrasts with an intelligent tutoring system (ITS) approach, which seeks to adapt content and sequence automatically to the student. The ITS vision is cited in LaRocca, Morgan, and Bellinger (this issue), and a related adaptive sequencing argument is put forth in Holland, Kaplan, and Sabol (this issue). These approaches incur heavy computational overhead, whereas we sought to focus investment in speech recognition and interface development.

Student Resources: Browsers and Talking Notebooks

Browsing capabilities supplement the choices offered by the VILTS. Keywords with translations and pronunciation of individual words by a native speaker and in a conversational context are available in all activities. In addition, as the last lesson activity (in the left-to-right sequence), a "talking notebook" is available. In the talking notebook, the student can review a list of lesson vocabulary items (words or phrases), see translations, hear the items spoken in isolation by a native speaker, and hear the items spoken in the various utterance contexts in which they appear in the lessons. Users can also add words with translations to this list, and the audio features automatically apply so that the word can be heard in isolation or in context.

A Communicative Approach: Listening Exercises and Dialogue Interactions

Pedagogical research in the VILTS project focused on how best to map the two speech technologies—speech recognition and automatic pronunciation scoring—into useful and valid lesson activities. The lesson interactions involving speech recognition were informed by the communicative approach to language teaching, under which cluster most of the popular methods that succeeded the audiolingual method. This approach is advocated in most government language teaching institutions as well as in public schools and universities. The communicative approach stresses the interactive use of a second language in a meaningful context with a high degree of comprehensible input, for example, exposure to the target language at as lightly higher level than the level at which a student is completely

389

comfortable (Krashen, 1981). To carry out this approach, our design had to emphasize the meaningful use of language for communication.

Following communicative considerations, the VILTS lesson architecture was designed to take advantage of the rich repository of authentic conversations we had collected by incorporating these conversations into a strong communicative framework. The emphasis in the listening mode is on (a) listening for the gist of a conversation to determine the main ideas and issues and (b) spotting key words and phrases. For example, in a phrase-spotting activity at the beginner level in ECHOS, the learner hears a short conversation and then is asked "What did you hear?" and sees a list of French phrases:

• tout le monde

• seposer

• peut-être

The learner clicks on the phrases heard in the order they occurred in the spoken segment.

The emphasis in the speaking and reading-aloud modes is on appropriate dialogue-like responses to specific cues that suggest conversational situations, given through photographs, graphics, and voice. For example, atypical speaking activity presents a situation and then directs the student respond by speaking a response from a menu of utterance choices. In a discussion about health habits, learners are asked about their own opinions and practices. To the question, "Do you smoke?" learners can choose to say one of three responses written on the screen:

• I smoke.

• I don't smoke.

• I used to, but I don't smoke anymore.

Any one of these phrases is a valid response, and the system highlights what the student is recognized to have said and issues a dialogue-like, nonjudgmental response. Alternatively, some choices are less appropriate to the question or cue. For example, in a discussion of health habits, the learner is asked about going to the doctor. The choices available are Je préfère aller voir ma famille 'I prefer to go see my family,' Je ne vais jamais chez le médecin 'I never go to the doctor,' and J'en ai un, 'I have one of them.' If the last response is recognized as the one spoken, the system highlights it and issues an expression of incomprehension. However, all response choices are structurally and grammatically properly formed because the focus of the program was not on the diagnosis and repair of structural errors.

390

At the same time that we stressed communication, we wanted to incorporate pronunciation evaluation. This is an aspect of language instruction not stressed in the communicative approach (Hammond, 1995) but one that we believe is essential to complete language learning (see, in this issue, Dalby & Kewley Port and Eskenazi).

Listening Before Speaking

Studies in second language learning suggest that the differences between production and comprehension are not as great as might be thought. For example, the beneficial effect of comprehension training on production is presented in Postovsky (1977). Krashen (1985), in discussing the benefits

of the "silent period" during which second language learners produce very little during initial exposure to language, argues that the competence of the learner is strengthened by the act of understanding. In the VILTS activities screen layout, the user is implicitly encouraged to begin with comprehension and discrimination activities. Speaking activities that involve shorter interchanges follow, and the last set of activities concentrates on longer segments of read speech to supplement the lesson with examples of prosody in longer text. The user is also encouraged to proceed from the top activity in each cluster to the bottom, moving from easier to more difficult activities. The top activity in each cluster presents the material used as a basis for the remaining activities in that cluster. This sequence, schematized in Figure 2, is suggested by the layout of the activities. This sequence is not imposed, however, and students are free to explore in various ways, as described above.

Figure 2

VILTS Screen Layout Suggesting (Not Imposing) a Lesson Activity Sequence

0x01 graphic

391

SPEECH TECHNOLOGY AND PEDAGOGICAL DESIGN

Speech is the first and most used linguistic medium, and speech technology enables computer-aided instruction to focus not just on text but also on speech. Students often have difficulty generalizing from the one, typically very careful, style of speaking that a teacher may represent to the many casual styles observed "in the street." We therefore thought it was important both to use spontaneous speech from a variety of talkers and to include more careful read versions to help bridge this gap. The use of unscripted, spontaneous materials, however, poses some challenges. Other major challenges have been the appropriate insertion of the technology into a pedagogical plan, the validation of the pronunciation scoring, and user evaluation. These challenges are discussed, respectively, in the sections below.

The Use of Unscripted Materials

Speech data collection represents a major task in our development of speech technology for language learning. We collected speech data from both native and nonnative speakers in order to train recognition models and to create scoring algorithms. In developing the ECHOS version of the VILTS, we also collected natural conversations from the same native speakers to serve as the basis for the lesson activities. Thus, the same speech samples serve to shape the recognizer and, inserted in the lessonware, to give listening practice. The conversations we collected were supplemented with samples of authentic text on similar topics read from the French newspaper Le Monde.

Our initial challenge came in gathering conversations at the three levels of difficulty called for in the VILTS architecture (see Figure 1 above). Conversations do not naturally fall neatly into these categories, nor do most conversations take place at the most basic beginning level. To address this issue, interviewers were trained in guiding conversations from beginning levels, with simple constructions and common vocabulary, to advanced levels incorporating greater linguistic sophistication. SRI collaborated with U.S. government instructors to gauge levels of difficulty and to provide appropriate sample questions to ensure that the conversations would be suitable for the leveled teaching framework standard in government instruction (e.g., we drew on "yes/no" questions to elicit beginning level conversations). The materials from Le Monde were simplified as appropriate for use in the lower levels.

How and where to incorporate these materials in the lesson architecture presented a further challenge. Natural, unscripted exchanges contain interruptions such as false starts, repeats, stammering, and other disfluencies,

392

as well as deviations from standard linguistic patterns. While this speech is very useful for training the student's ear to real interactions, the more carefully spoken, read speech is used in the system as the model for student speech. In the VILTS, the unscripted conversations were mapped to the listening comprehension exercises so that the user would become accustomed to the flow of authentic conversation with all its natural disfluencies and its linguistic and prosodic variations and irregularities. However, in the subsequent modes of speaking and reading aloud, the clearer, read version of the conversation was used as a model when the user was asked to produce speech. In some activities, such as role playing or question posing, a combination of the spontaneous and the read versions was used to mimic more closely natural interactions. In these exercises, the user's turn was modeled by the careful speech, and the system response was the conversational version. Finally, the text from Le Monde was used as a model in the reading-aloud mode to incorporate prosody models for longer segments of speech.

Speech Recognition in a Pedagogical Plan

A major challenge in the VILTS project was to develop a system that was both technically feasible and pedagogically valid. The communicative framework we chose to follow emphasizes meaningful interactions in the target language. However, as Hammond (1995) points out, communicative theory does not make explicit claims about the teaching of pronunciation, and, by its omission, accords pronunciation a less important status. Spoken interaction with a minimum of textual support is a major feature of the communicative approach. However, a system in which student speech was to be evaluated and a valid score returned seemed to require text on the screen to be read by the student. Speech recognition technology, although it has made remarkable strides in recent years, is still far from being able to understand arbitrary speech. Spontaneous speech from native speakers is quite a challenge, and nonnative speech is still more challenging. We decided that the desire for score validity should take precedence over the desire for spontaneity when pronunciation was to be scored. Therefore, we developed initial lesson activities in which all pronunciations being scored were based on reading from text. Lesson activities in which pronunciation is not scored can be more flexible, as will be discussed later.

Although communicative theory relegates pronunciation practice to a relatively minor role, this view has been controversial (Hammond, 1995), and even some proponents agree that evaluation and instruction would be beneficial to many learners. A central issue in pronunciation training is to determine the possible role of explicit instruction (Terrell, 1989). There is

393

VILTS: A Tale of Two Technologies

also a body of research showing that adult learners are capable of perceiving, imitating, and learning new phonetic distinctions (e.g., Flege & Hammond, 1982; Hammond & Flege, 1988; Rochet, 1993). A study by Catford and Pisoni (1970) examined auditory versus articulatory training in English-speaking students learning "exotic" sounds such as the glottal stop and a glottalized "k." Based on their test results, they suggest that ear-training and mimicking alone, while effective for some students, are less effective in general than articulatory training in teaching both auditory discrimination and the production of exotic sounds. Other studies with Japanese students learning to identify English /r/ and /l/ offer additional evidence indicating that the knowledge gained during perceptual learning may be transferred to production (Bradlow et al., 1995).

We developed an architecture for the VILTS that would admit a communicative approach in which student responses were elicited as meaningful utterances in context. The architecture includes separate modules to review student progress as they go through lessons; these modules consist of pronunciation evaluations after each lesson on overall pronunciation and individual sounds. While students are engaged in communicative activities in the lessons, the recognizer guides the interactions with appropriate, conversation-like responses from the system. Simultaneously, student input is logged and stored as a basis for pronunciation evaluation, but a score is established only after a lesson is completed and sufficient speech has been collected to return a reliable score. Since it is desirable to collect multiple student utterances to ensure consistent and reliable pronunciation evaluation, the background collection of input during communicative activities—before returning a score at the conclusion of a lesson—supports both the communicative component of learning activities and the robustness required of the scoring technology.

Because we also envisioned comprehension activities with no text as a crutch, the system was designed to begin with listening comprehension and discrimination activities and to flow into activities eliciting speech from the student. A range of activities was developed: those based on spoken material alone, those combining speech and text, and those based largely on text. This approach is seen in the trimodal architecture of the lessons containing sets of listening, speaking, and reading-aloud activities (see Figure 2 above).

Another issue in research and implementation of the French speech recognizer in the VILTS was that of trade-offs in weighting types of possible recognition errors. The main types of potential recognition errors are misrecognition (or false acceptance) and false rejection. When the rejection weight is high, the rate of false rejections is high, but the rate of false acceptances/misrecognitions is low. On the other hand, lowering the rejection weight results in more frequent misrecognitions/false acceptances but less frequent false rejections. Research was conducted on the optimum

394

strategy for pedagogical purposes. We tried to minimize both false acceptance and rejection. However, we biased the system toward rejection because we felt that forcing a student to repeat a marginal utterance was preferable to the risk of falsely accepting an error and potentially confusing the student.

Validation of Automatic Pronunciation Scoring

In our scoring paradigm in the system, both native and nonnative speech data are collected, and a database is created of ratings from human experts (e.g., language instructors) to support the development of machine scores. We use the data to assess the reliability of human ratings of pronunciation and to develop and assess pronunciation scoring that correlates well with humans. We treat pronunciation evaluation as a prediction problem: Can we predict the score a human expert would assign to a particular speech segment? Using the speech and the expert-ratings data, we build statistical models and assess various machine scores as predictor variables.

For the VILTS and related systems, we developed and tested our algorithms on data collected from native speakers of American English speaking French and Spanish. In the ECHOS project, the human experts were senior French instructors at a major government language school. In the Spanish project, the raters were a panel of five native Spanish speakers. We showed that, when a sufficient amount of speech data are available (many student utterances over the course of a lesson), certain machine scores(e.g., the log-posterior and the normalized duration) achieve a correlation with the human scores comparable to the correlation between human raters. Thus, the agreement between human raters serves as a benchmark for the degree of agreement desired between automatic scores and human judgments. Note that our scoring is intended to operate on multiple-word utterances containing arbitrary words without the need to collect additional data. This situation is far different from systems trained on specific minimal pair input such as the pronunciation trainer described by Dalby and Kewley-Port (this issue) in which pronunciation scoring has been found to proceed reliably with smaller samples of speech.

Speech recognition technology is key to the automatic evaluation of pronunciation quality. However, standard speech recognition algorithms were not designed with the goal of pronunciation scoring. Therefore, new methods and algorithms had to be devised to match the perceptual capabilities of human listeners in scoring speech quality. The factors affecting human scores(and therefore our automatic scores) include duration, syllabic timing, spectral, and posterior scores. Best results are obtained by combining several scores. We combined scores using neural networks and classification trees, and we also used Bayesian approaches. Further improvements

395

were achieved by modeling intraword pauses in nonnative speech. By using these scores, we were able to obtain human-machine score correlations comparable to human-human correlations on the same data set.

For example, for our Spanish data, the panel of native speakers rated the overall pronunciation of each nonnative sentence on a scale of 1 to 5, ranging from "strongly nonnative" to "almost native." These human scores were used to evaluate the effectiveness of the machine scores and to calibrate the mappings from the machine scores to the predicted pronunciation scores. To assess the consistency of these human scores, two types of correlation were computed. At the sentence level, pairs of corresponding ratings for all the individual sentences were correlated. At the speaker level, the scores for all the sentences from each speaker were averaged, and then the sequence of pairs of corresponding average scores for each of the speakers was correlated. The correlation between raters was computed on a subset of 2,800 sentences rated by all raters. The average sentence/speaker-level interrater correlation was.68/.91. Our best sentence-level result using automatic pronunciation scores was a correlation of .609, very close to the average .68 correlation found between human raters. Details of these results appear in Neumeyer et al. (1998).

User Evaluation of the VILTS

Initial user evaluation of the VILTS prototype was begun after the French recognizer was judged sufficiently robust to support smooth interactions. The first lesson was used as the test bed for the evaluations. A questionnaire wa sdeveloped, focusing on a qualitative analysis. Five subjects were interviewed in three sessions, each lasting approximately two hours. Table 1 summarizes the demographics of the subject population.

Table 1

VILTS Evaluation Subject Data

0x01 graphic

396

Subjects were provided with written instructions describing the goals of the project and the current state of development. They were informed that the project was not targeted to structured grammar learning but rather to the elicitation of speech in an engaging and interactive environment to support pronunciation scoring and feedback. The questionnaire was presented orally, and the interviewers encouraged dialogue and comments throughout lesson use. Subjects were asked about their likes and dislikes with respect to each activity and to the program overall. They were assisted in working through the lessons only when they seemed stalled or frustrated. Sessions were recorded and transcribed for analysis. Although project resources permitted evaluation with only one lesson, the results were illuminating as a guide to future refinements as well as to interface design for language learning in general. In some areas all subjects agreed, and in others they had divergent opinions, often seeming to depend on their language skill levels. We focused on documenting system strengths and areas needing future work.

The interviewees noted the following strengths of the system:

• They all reacted very positively to interactions with authentic, unscripted materials because they felt that the point of learning a language was to converse in real-life situations. The unscripted nature of the conversations made the activity resemble a real-life situation.

• They liked hearing native French without much knowledge of the content and trying to figure out what was being said. Stronger speakers enjoyed the challenge of trying to comprehend without seeing text; weaker speakers wanted an option to display the text.

• The incorporation of high-quality speech recognition proved to be a major strength for all subjects; an interactive activity that elicited spoken French from them was an important factor in their enthusiasm.

• Most important to all subjects was the high quality and real-time nature of the interactions that mimicked real-life interactions.

• Very important to all subjects was the ability of the system to readily recognize nonnative French input and to reject poor pronunciations. They found that when their pronunciation was rejected, they were able to consult key words, practice and compare with a native speaker until they were more confident, and then return to the program and continue with improved pronunciation. The subjects also noted that practice with the key

397

words helped them learn the vocabulary. They found that they used the key words less often as they learned more, which afforded them a high level of satisfaction.

• The key word and browsing capabilities were also well received, particularly the ability to hear the words in context. The subjects with weaker French ability requested the addition of more such key words in the system.

• The self-paced nature of the program was another highly appreciated system component. Subjects liked the ability to navigate freely through the activities and to repeat or review as needed. Since loudspeaker buttons were available to review native pronunciations of all system material as often as necessary, subjects took advantage of this feature to suit their level of comfort and to practice until they felt confident enough to move on. "You could work with it until you understood it" was a comment often heard in the feedback.

• Interviewers noted that four of the five subjects asked if they could return on their own time to use the system. The fifth, the weakest in French ability, felt that the system was more challenging than suitable to his level.

The interviewees made the following suggestions for improvements:

• More support mechanisms are needed at various levels. The subjects wanted to seethe VILTS include video clips, an on-line dictionary, and expanded hypertext capabilities. Since only one lesson was available for evaluation, they felt that a greater range of levels should be accommodated. Concomitant with this recommendation was the request for shorter segments of material for users with lower French ability, along with more textual support. The subjects also asked for more help in repairing pronunciation.

• Although the system is highly navigable, subjects would have liked even more flexibility, for example, to move backwards within an activity. They would also have liked the ability to determine the level of pronunciation skill required. Future research is needed to support this function robustly.

• Subjects suggested tracking mechanisms, practice spaces, and a timing feature. They wanted to see indications of their progress through the lessons and also to see how much of an individual activity they had completed. Some expressed a desire to practice both responses and pronunciation before embarking on a

398

scoring session. Finally, some of the more advanced subjects thought that proceeding through the activities against a timer would encourage them to improve their fluency in French.

We observed that, in general, the subjects needed some exploration and ramp-up time with more than one activity to become accustomed to any interface.

FUTURE DIRECTIONS

The initial user evaluations suggest several future directions. Clearly, more research is required to understand how technology can improve language learning under different circumstances and for different types and levels of learners. We need to understand differences in short-term versus long-term improvements and how these improvements are affected by different practice and feedback methods. Future directions are also informed by the ongoing research in speech technologies. We believe that the future will continue to require multidisciplinary interaction between those representing language pedagogy and those representing speech and language technology. Below we outline directions in terms of additional lesson development, followed by pronunciation scoring and feedback.

Additional Lesson Development: Toward Free-Form Utterances

The unscripted French conversations we collected for ECHOS could support additional lesson activities. Our research suggests that the most expeditious next step might be to develop new lessons with a narrower

scope (e.g., fewer than 15 activities per lesson). The activities would be selected on the basis of pedagogical impact relative to development effort. In addition, excerpts from conversations not yet used could provide a greater breadth of exposure without the overhead of developing coherent, linked materials around full conversations, some of which are more interesting or easier to work with than others. Students could select lessons according to a topic of interest, or lessons could be indexed for pronunciation or linguistic form. This approach would allow rapid development of new lessons, support additional French speech production by students, and provide a rich environment for interactions using a variety of French speakers and speaking styles. The greater the number of lessons available to support student practice and speech, the greater the opportunity to foster improvement and track progress.

We have investigated some of these ideas in a new language learning application that leverages the components and architecture of the VILTS.

399

This program, called Special Operations Language Voice Interactive Training (SOLVIT), was developed in just a few months by reusing VILTS components and adding new activities. In SOLVIT, the student is coached through successively more independent spoken interactions in French to a level of free-form utterances bounded only by the types of constructions and vocabulary introduced in the lesson. Students produce these utterances without text support and without reading utterance choices from the screen. To foster the predictable utterances needed by speech recognition, SOLVIT coaches students on target sentence elements in early speaking activities and then encourages them to recombine these elements to create new utterances in later activities. SOLVIT also uses the principle of graphically displayed artifacts to guide students' utterances. Normal artifacts of human interaction—desk calendars, restaurant menus, office checklists, and road maps—constrain what we talk about in ordinary conversations; in a language lesson such elements serve to limit what the learner says. Figure 3 shows this principle at work in a sample SOLVIT screen, which displays a map and a checklist for assessing the conditions of supply routes from an airfield. The student must ask about the routes shown on the map and assess the features (shown as icons) on the checklist.

Figure 3

Sample SOLVIT Screen Showing a Task to Evaluate Road Conditions Using Spoken French

0x01 graphic

400

The artifacts shown in Figure 3 appear as a natural part of the lesson scenario: to provide relief to a hurricane-stricken Caribbean island where the inhabitants speak only French.

Pronunciation Scoring and Feedback

Pronunciation scoring, as we saw in the initial user studies, provides useful feedback to the learner. It has the potential of being even more useful if we can devise ways to provide more detailed feedback, diagnosis, and repair strategies. In selecting feedback, it is important to understand both technical challenges and pedagogical validity. Future directions include the following:

•Sentence/Sounds Imitation

Increased exposure to French native speech, together with imitation and repetition, has been a basic technique used in language classrooms. Although commercial systems sometimes return pronunciation scores (see Wachowicz & Scott, this issue), it is unclear what those scores mean. Based on our research to date, we estimate the need for about 30 sentences from the learner to provide a valid score, that is, one that correlates well with human experts. As research on pronunciation scoring and feedback progresses, it may be possible to return a valid score with fewer utterances. We can then envision exercises in which the student practices on first short and then incrementally longer utterances. With such a system, students could practice and compare their speech with that of a native speaker as often as they wished, at their own pace, and receive feedback on the effectiveness of this method in improving pronunciation.

• Formative Feedback

Formative feedback to improve student pronunciation could be provided through various graphical representations. Representations of the speech production apparatus (lips and vocal tract) and/or the speech waveform could be used to compare the student's speech with that of a native speaker or to illustrate a student's pronunciation problems. Many displays are possible and several are now in use. However, little data are available showing whether such displays actually improve pronunciation, and, if so, whether students can generalize to conditions of normal interactions when such feedback is not available. We recommend research aimed at assessing whether feedback improves pronunciation and whether any gains carry over to conditions

401

where no feedback is available.

• Speech Representation

There are several ways of representing the speech segment relevant to pronunciation training, including speech waveforms and spectrograms (see Eskenazi, this issue). These representations can be displayed along with corresponding phonetic transcriptions and word alignments. An interactive system that allows users to click on certain regions of the waveform and hear them could be developed. Similar displays of native speakers could be presented to illustrate differences between student and native speech. Scores might also be color-coded to represent the different phonetic segments so that students could determine where their areas of difficulty lie. Since many of these displays (especially spectrograms and phonetic symbols) incur significant training costs before students can make use of them, research is needed to understand the costs and benefits of teaching these skills.

NOTE

1 All product and company names mentioned in this paper are the trademarks of their holders.

REFERENCES

Bradlow, A., Pisoni, D., Akahane-Yamada, R., & Tohkura, Y. (1995). Training Japanese listeners to identify English /r/ and /l/ IV: Some effects of perceptual learning on speech production. Progress Report No. 20, Research on Spoken Language Processing. Bloomington, IN: Indiana University.

Catford, J. C., & Pisoni, D. (1970). Auditory vs. articulatory training in exotic sounds. Modern Language Journal, 54 (7), 477-481.

Flege, J., & Hammond, R. (1982). Mimicry of non-distinctive phonetic differences between language varieties. Studies in Second Language Acquisition, 5 (1), 1-17.

Hammond, R. (1995). Foreign accent and phonetic interference. In F. R. Eckman (Ed.), Second language acquisition theory and pedagogy. Mahwah, NJ: Lawrence Erlbaum.

Hammond, R., & Flege, J. (1988). Attitudes, experience, and the mimicry of sounds: Implications for second language acquisition. Paper presented at the Seventh International Symposium on International Perspectives on Language, Literature, and Culture, George Mason University, Fairfax, VA.

402

Krashen, S. (1981). Second language acquisition and second language learning. Oxford: Pergamon.

Krashen, S. (1985). The input hypothesis. London: Longman.

Lee, J., & VanPatten, B. (1995). Making communicative language teaching happen. New York: McGraw-Hill.

Neumeyer, L., Franco, H., Abrash, V., Julia, L., Ronen, O., Bratt, H., Bing, J., Digalakis, V., & Rypa, M. (1998). WebGrader(TM): A multilingual pronunciation practice tool. In Proceedings of the Workshop on Speech Technology in Language Learning, Stockholm, Sweden.

Oxford, R. (1995). Linking theories of learning with intelligent computer-assisted language learning. In V. M. Holland, J. Kaplan, & M. Sams (Eds.), Intelligent language tutors: Theory shaping technology. Mahwah, NJ: Lawrence Erlbaum.

Postovsky, V. A. (1977). Why not start speaking later? In M. K. Burt, H. C. Dulay, & M. Finocchari (Eds.), English as a second language. New York: Regents.

Rochet, B. (1993). The role of auditory training in teaching nonnative speech contrasts. Paper presented at the Third Conference on Second Language Acquisition and Foreign Language Learning, Purdue University, West Lafayette, IN.

Terrell, T. (1989). Teaching Spanish pronunciation in a communicative approach. In P. Bjarkman & R. Hammond (Eds.), American Spanish pronunciation—Theoretical and applied perspectives. Washington, D. C.: Georgetown University Press.

ACKNOWLEDGEMENTS

We thank Leo Neumeyer and the SRI algorithms team for their work on the speech technology components of VILTS, as well as the U.S. Government language teachers and developers who collaborated on graphical design and language pedagogy and who also provided many of the ratings of pronunciation. We also thank Harry Bratt of SRI, who led the SOLVIT effort, and the Defense Advanced Research Projects Agency for its support of SOLVIT, as well as researchers at the U.S. Army Research Institute and staff at the Special Operations Forces Language Office for collaborating on the content of the SOLVIT lesson. We especially thank Melissa Holland of the U.S. Army Research Lab for extensive useful comments on this paper.

403

Language and Computing at Duke University: Or, Virtue Triumphant, for the Time Being

Frank L. Borchardt

When we honor John R. Russell we honor him, to be sure, and a handful of others of his generation who opened this new field and began or advanced its cultivation. Appearances to the contrary notwithstanding, this is not a wholly selfless activity. We are drawing maps by which to orient ourselves, to locate ourselves in time and the metaphorical space of our hard work. When we honor any pioneering individual and individuals we are, in the same gesture, defining our common field of endeavor. Because our work is new, our reflection upon it will also incline to be new. Its history will not provide, by itself, easy and obvious plots on which to hang the facts.

It turns out that the present writer spent eight years of his education in the care of the Jesuit Fathers. These years left indelible patterns imprinted all over his mind, among them, an inclination to look at events in time less as a continuous narrative than as a morality play. This morality play has white hats and black hats, cattlemen and settlers (can the cowboy and the farmer ever really be friends?) and rustlers, lawmen and gamblers and outlaws. The values of family and the virtues of civilization always win out in the end, and the white hat always triumphs. It is said that history is written by the victor. That is true, of course, but only when the contest has been decided. For

57

contests not yet decided or never to be decided it is perhaps truer to say that history is written by the historian. The historian determines what meaning, if any, the course of events in time is going to have. Unless someone else comes along and rewrites this plot, the story of computing and languages at Duke University is going to be a morality play, in which, for the time being, Virtue is triumphant.

CALIS and Method

Sometime near the end of the 1970s, Leland R. Phelps, Professor of German at Duke University, and author of several widely used German readers,1 took notice of some CAI going on in another environment, the big introductory Economics course at Duke, conducted by Professor Allen Kelley, and thought: there's no reason we couldn't do the same thing in German. Without asking anyone's permission or consulting particularly with his colleagues Professor Phelps gathered together funding from sources at Duke (ultimately the "Commonwealth Fund") and outside (the office in Atlanta of the Consul General of the Federal Republic of Germany). With one part of this funding he supported through the M.A. a graduate student, K. Omar F. Hossain, who wrote the code for the first realization of CALIS (Computer Assisted Language Instruction System). The program was written in HP-BASIC for the Hewlett Packard HP-2000 minicomputer, located remotely in Research Triangle Park, NC, and communicating sometimes directly over dedicated lines, sometimes over phone lines connected at 300 baud to ADM3 and later ADM5 monochrome dumb terminals in the language labs. With the remaining funds, Phelps supported one of the colleagues in the German Department, Professor Helga Bessent, with a leave of absence so she could write the content, a dataset that followed chapter by chapter the elementary German textbook of choice that year. In its first incarnation, the program allowed a one-line question and one one-word answer. Care was taken to change the wording from the textbook so as not to violate copyright (Hossein, Phelps, and Bessent 1980).

In very short order it became painfully clear that this was not going to be flexible enough. Although the students took to the program at once and very favorably indeed, they objected vigorously to what they believed to be mistakes made by the program.2

58

This was especially embarrassing when they were right. When that was the case it was usually not because "the" right answer was mistaken or mistyped but because there was more than one right answer. The template written for authoring was immediately expanded to allow as many as five right answers.

No one realized it at the time but three critically important principles were being shaped in this episode: all exercise forms and the critique of exercise forms reflect theoretical considerations, whether conscious or covert, in the electronic medium; not only the content but also the delivery mechanism is infinitely revisable; corollary: there is no closure, the end users, teachers and students, determine how the program and the content are going to be revised; corollary: our technology will be user driven.

The theoretical considerations underlying the "one-right-answer" structure in the delivery mechanism were as follows: "one-right-answer" might be perfectly acceptable for certain kinds of learning, and if not learning, then for certain kinds of training, specifically, highly structured and optimized procedural knowledge, like learning to tie a tourniquet. Adult language learning was not perceived as fitting those categories. "One-right-answer" was felt to distort the realities of language learning as these teachers were convinced it occurred in practice.3 Multiple possible right answers at one blow asserted the premise that language was constructively redundant and rejected the thesis that a human language was a one-on-one mapping of one language on another, or the one-on-one encoding of one code in another. One layer lower, rejection of a "one-

right-answer" structure denied any known language the privilege of being the "real" language, of having the power of accurate, precise, or "true" one-on-one encoding of "reality."

The relatively easy revisability of the whole enterprise led to imminent changes on a large scale, first to the delivery engine. The textbook specific content remained largely the same for a while, except for constant corrections in detail. The time and labor invested in the electronic dataset and its obvious unsuitability for other textbooks with other emphases postponed the change of adopted textbook for several years. However, the delivery mechanism, CALIS itself, underwent a major transformation immediately after the first experiment. It was express dissatisfaction with the rigidity of the first version that made the revision logical and sensible, and it was the users' comments which determined the direction of the revision.

59

Professor Phelps sought out a Duke alumnus, Thomas B. Clark, III, at the time a medical student at the Medical University of South Carolina, and sometime organist in the Duke Memorial Methodist Church, when he was not rewriting CALIS with a grand new vision. Clark wrote authoring templates for short answer (with error analysis), long answer, true-false, multiple choice, reading comprehension, and sentence combining exercises. He reconceived the answer judging mechanism to include "wild cards," asterisk (*) for "anything" and the “exclusive or" (XOR) function within curly brackets {} separated by vertical bars {|} for choices among alternative right answers. With these added functions it was already possible to provide a kind of pseudoparsing for longer answers, especially considering the syntactic regularities found in elementary German. He developed routines for dividing the screen horizontally and allowing text to remain in the upper half (or be scrolled through), and questions to appear sequentially in the lower half. Randomized "praise/blame" feedback gave the machine what passed then for a semblance of humanity, while it gave concrete reality to the behaviorist presuppositions of the method.

Using the programmer's initiative Clark also provided MAIL functions, so that teachers, students, programmer, and systems administrators could all communicate one with the other. This was intended chiefly to make revision as efficient as possible, so that complaints could be recorded as their causes occurred, and the right person contacted to make the correction. A small number of "global" linguistic functions were included in this edition of CALIS, including a SYNONYM function, by which such regular equivalents as "zu dem=zum" would be allowed from the very outset across all CALIS content. This was the ancestor of such later global CALIS functions as describing rigorous and lax answer judging, depending on whether the author was more worried in this activity about raw spelling than accurate diacritics. Likewise a DICTIONARY function was being developed that was to allow a student to retrieve an English equivalent for any unrecognized German word, and thus to lessen the trauma of abandoning one textbook for another. The assumption was still that German grammar was German grammar, and that therefore the only substantive differences from one textbook to the other would be vocabulary. The notion that methodologies were undergoing convulsive revolution had not yet dawned on most of those involved, even

60

though several of them were implicated in those very changes: lexical solutions were not alone going to render datasets compatible. All of this was still taking place in HP-BASIC, on a remote HP-2000 minicomputer, and on ADM3 and ADM5 terminals, for which Clark had also to write terminal identification programs, record keeping, and housekeeping functions.

One colossal failure of this phase was the testing module. Clark developed a whole mechanism by which a midterm examination could be composed, scrambled randomly for security purposes, delivered, and automatically graded. The degree of anxiety that this generated among students and faculty alike was wildly beyond anyone's reasonable expectations. First of all students and faculty both demanded that the experiment be backed up by a print version that was actually given in class in addition to and after the electronic version. Then student performance on the electronic version was deliberately so erratic as to make the results useless not only for grading but also for scientific study. The experiment was abandoned altogether. The episode proved the following: that objective desirability of a feature and technology driven ease play no role in realization of a feature. If the feature is not initiated by or demanded by end users, it is in serious trouble to begin with. Take this instance: what is more excruciatingly time consuming than the quizzing and testing aspect of college language teaching? What could possibly be more desirable than its automation? If the end users fail to agree to this proposition, if they cannot be persuaded of the feature's utility, then it is, quite simply, doomed.

CALIS and the Industry

The phase of activities just described runs approximately from 1979 to 1982, when the development of CALIS was chiefly an internal affair. The funding was reasonable and could be accomplished alongside the normal funding of teaching and research in a relatively small academic German department. The setting in which CALIS would develop outgrew the home front in the academic year 1982/83 with a series of external events. These began with a meeting of the South Atlantic Modern Language Association (SAMLA) at which there was an unusual, all-morning panel discussion in the grand ballroom on computing in language and literature. It featured Doug Short and Sarah

61

Burton of North Carolina State University and John B. Smith, then of Penn State, all of whom played a role at a meeting at North Carolina State University in Raleigh in June, 1983, which represented a huge step forward in the visibility of computing and humanistic work, including the visibility of computerized language instruction.4 At the first of these meetings individuals in the IBM organization first paid attention to what was going on in the CALIS project. A rather spectacular and far from conventional IBMer, Irene Copley, took the lead, brought a display of personal computing down to Duke, held out all the golden prospects which an IBM XT would open, far beyond the capacities of the original IBM PC.

CALIS still lived on an HP-2000 minicomputer, located remotely at Research Triangle Park (RTP), NC. This very situation permitted a sensational display of CALIS at the 1982 Modern Language Association meetings in Los Angeles. It was the first time that IBM had appeared at the MLA meetings in force in the exhibit area. The unifying idea was: how many different ways could you use PCs to help do the work MLA members do. CALIS could not run natively on an IBM PC, but it could be made to emulate a dumb monitor. So, with an acoustic coupler, a 300 baud modem, and a hotel phone line, one PC was hooked up to the HP-2000 in Research Triangle Park, NC, and conventioneers could work their German CALIS exercises at a 3,000 mile remove. The ten year old son of one professorial colleague judged the program satisfactory and so made the enormous effort worthwhile.

Negotiations between the CALIS project and the computer industry began at this time, determined first by the needs of connectivity. Classroom based computer assisted instruction demanded the ability to perform centralized record keeping. In a real sense, this imperative has not changed, even in the days of desktop minis. In 1982, it was still hard to imagine distributed computing power, without the need of a powerful central computer and much less powerful, indeed dumb delivery stations on the periphery. The PC revolution was, however, making itself felt, so that the space seized by the dumb station was envisioned as equally well or even better occupied by some sort of freestanding personal computer that could act as the servant of a mainframe or large minicomputer located remotely.

62

The result of that particular configuration was negotiation with the Digital Equipment Corporation for an installation that would feature at its center a VAX 750 and at the periphery a dozen or two DEC PCs, either Rainbows or Professionals. It was a moment when competition in the desktop computer market was at its fiercest and developments at their fastest. The configuration looked like a good idea, at least for German and other western European languages.

Character Sets

The outstanding problem was the other-than western European languages with their non-Roman character sets. This problem expressed itself most audibly in the complaint of a good colleague in the Classics Department, Professor William H. Willis. He edited a journal, GRBS (Greek, Roman, and Byzantine Studies), which required Coptic as well as Greek, Hebrew, Arabic, and the occasional hieroglyph. His printer was about to double or treble the price of laying out his journal in type, which would have caused the journal to go under. Professor Willis came by and suggested from our experience with CALIS, we ought to be able to find a way of solving his typesetting problem at a much lower price than his printer was demanding. This was sometime in early 1982. Together we surveyed the field, which was precious narrow at the time. The most flexible system seemed to be a Cyber 37 mainframe at the University of Texas using Tektronix 45 graphics terminals, on which Arabic had been taught for some years. This was hardly cheap enough to solve the current problem or fresh enough to avoid the likelihood of imminent obsolescence. By happy chance, the problems specific to classical languages had been addressed by the scion of the house of Packard (as in Hewlett Packard), David Packard, Jr., in a system he named after his cat (in turn named after the murdered Greek poet) Ibycus. The system arrived in Professor Willis's project late in 1982 and more than solved his publication problems. It brought him and his colleagues into a major project, the compilation on-line of the totality of papyrus documents available in print.

63

No such ready solution awaited the rest of the non-Roman character sets. The DEC constellation did not address the problem, but a new competitor in the field, the Victor 9000, did. This largely forgotten but altogether wonderful teaching machine came with a 6OOx8OO pixel monitor and a capacity to redraw the character set in character mode, to allow, say, Russian and English on the same line. It also provided an audio interface at some ridiculous price like $150. It was the ideal language learning machine, well ahead of its time, anticipating EGA standards in the monitor and Macintosh-like functionality with peripherals like audio. The Victor 9000 served our colleagues who needed Cyrillic, Greek, Coptic, and Hebrew character sets for years after the bankruptcy of the company and well into the era of the Macintosh, which substantially addressed those problems for many languages.

Experience with the Victor 9000 provided invaluable precedents for the work the CALIS project would have to do over the next few years. Much of the early work in Chinese undertaken by Dr. Richard Kunst employed the Victor 9000 as the platform. A Russian overlay was developed for the Victor 9000, which, when hooked up to the HP-2000 in RTP, would present Russian CALIS exercises properly in Cyrillic typeface. This stayed in active student use for several years until the phasing out of the Victors. The program provided at least one comical moment at the 1985 CALICO meeting when failure to find a rentable phone line led CALIS project folk to try to hook the computer to an acoustic coupler and from that to a pay phone in the lobby, thus to connect back to North Carolina. It provided an instance of the hardware disaster par excellence. The fate of Victor was not to be settled for a couple of years, and until that very moment, and even beyond, Victor seemed like a good solution for the language presentation problem.

Two additional alternatives for the character set problem appeared in these early years or shortly thereafter. The one was the Xerox Star Station, which came on loan with a huge hard disk filled with Kanji, the Chinese characters employed in Japanese, and an IBM 5550, dedicated to Japanese and in use until this very year as a word processor. Both represented, in themselves, technological culs-de-sac, even though the Xerox Star was soon to enjoy a whole new incarnation as the Apple Macintosh.

64

The critical development that made all the difference early, and first opened the way toward standardized, low cost computing in foreign character sets was the publication of the EGA standard and an industry ready to develop from there forward. In the wake of this development, a programmer came to the Project from the study of religion, Jeff Gillette, one who understood the needs of his teachers and colleagues when it came to the character set issue. He designed a program known as the Duke Language Toolkit by which alphabetic characters could be drawn in any number of alphabets or syllabaries, summoned to override the standard keyboard, and applied within any program that did not itself seize the keyboard. The Project had a solution for its character set needs, one that would last as long as the character based screen survived. The graphical user interface (GUI) changed all of that, yet again.

CALIS Rewritten for the IBM PC

The relentless acceleration inherent in technology has made us get ahead of ourselves. The EGA and GUI, coming as they do in the mid to late eighties, skip over several years of critical developments, chiefly at home, but also in the great world outside the university. During the explorations undertaken in 1982, it became clear that any approach to the local administration involving many hundreds of thousands of dollars would not be taken seriously if it came on the initiative of a small language department alone, German, or even of two, German and Classics. A project was therefore conceived involving a good many departments that had in common the need to manipulate language one way or the other. We more or less defined the Humanities as those disciplines that employed language as the central too], perhaps also the central object of their investigations. Rather than engage in a fruitless controversy on the definition of the Humanities, we called the project "Computerization of Language Oriented Enterprises" (COLOE) and brought on board colleagues in substantial numbers from Asian and African languages, Classics, English, Music, Religion, Slavic, the Divinity School, the university library, and the university press. This project requested a budget of $750,000 and foresaw a solution generally on the lines of the DEC configuration.

65

The COLOE proposal envisioned a pyramid of users:

• At the base, students in the study of more frequently taught modern languages chiefly in Roman alphabets;

• Thereupon, students engaged in the study of less frequently taught languages, chiefly in non-Roman alphabets;

• Thereupon, clerical users in those humanities departments employing the above mentioned character sets;

• Thereupon, professorial research in the humanities exploiting those character sets;

• At the peak of the pyramid, publication, that is, the ability to produce camera ready copy in all the above character sets.

The basic idea was to grab those who held the purse strings where their hearts and minds were the softest, undergraduate education, and to finance their needs generously, while allowing humanistic research to ride on the resources generated for the students.

It is possible that language education was excluded from the sentimentality of the moneybags. In any event, the proposal encountered precious little softness of heart and none of mind. It rattled around the administration for over a year. That particular fact is remembered because a letter went to the university president saying:

Dear Mr. President,

I observe with melancholy the arrival of the first anniversary of the submission of the COLOE proposal and, with it, the gradual departure of a golden opportunity for Duke to take leadership in an important and interesting new field.

Sincerely yours,

Frank L. Borchardt

In the wake of this letter, the DEC configuration started making its way through the decision process. It was on the verge of success when a powerful competitor intervened with a promise of substantial largess if this sale were scuttled. And scuttled it was. To be fair to the competitor, the official computing establishment was looking for any possible excuse to prevent that kind of money from going into a new and untested area.

66

It was only too delighted to have an external justification for recommending against the purchase.

As a consolation prize another solution was sought by the administration and found. The President, Terry Sanford, past governor of the state of North Carolina and later to be U.S. Senator, together with his Chancellor and soon-to-be successor in the presidency, H. Keith H. Brodie, looked across the discretionary accounts of the university, found a three year computing project coming to an end, and proposed to the Duke Endowment-a philanthropy that supports Duke University but is separate from it - three year funding for the Humanities computing projectattherateof$100,000ayear. This bird in the hand somehow compensated fully for the several birds in the bush represented by the DEC prospect and the competitor's promise of largess. It permitted us to concentrate on getting computers on the desks of colleagues in the humanities and to work on getting the best possible CALIS before students in German and Russian.

With this substantial funding in hand the major acquisition the Project was able to grasp was in personnel: a wandering scholar with unusual experience in computers even from his graduate student days, and with a Ph.D. in German from the neighboring University of North Carolina at Chapel Hill, happened by at just the right time: Dr. Peter Batke. With a full time deputy on hand to make house calls, to run things day to day, and to keep abreast of developments, the project was able to take off in several positive directions. The first and most visible of these was in the area of text accumulation through optical character recognition (OCR). The Pew Memorial Trust had just provided a major grant to the university, shepherded to success by a new assistant to the Dean of the College, Dr. Lee Willard, a Classicist. The grant was specifically for the purpose of equipment. That the purpose was actually scientific equipment for scientists was not made clear to the likes of us in the project, and we applied for the funding of a Kurzweil Optical Character Scanner. The price was in the upper five figure range and, astonishingly, our application was granted. All of this transpired in the spring of 1983. The machine came with an enormous stand-up reel-to-reel tape storage device, an H-P graphics terminal, a scanning surface, and an exchangeable eighteen inch five megabyte hard drive for the storage of troublesome data such as the Cyrillic character set for scanning Russian. It was a vastly more

67

impressive and photogenic installation than its modern successor, which can easily accompany a normal desktop computer for far less than the price of one month of the Kurzweil service contract, and do everything, and more, that one of those mastodons could do twelve years before.

Some serious work came out of the ownership of that machine, for example, a rather good M.A. thesis on the shift in Schiller's writings from clinical metaphors rooted in his medical training to a broader, less predominantly materialistic palette.5 The machine was put at the service of the university at large and many departments took advantage of the newest in OCR, including projects in the medical division, where the transformation of typed and printed records to on-line searchable data was an obvious and pressing need. On one occasion, the Provost's office succeeded in losing weeks of work on their word processor but, thank heavens, had a hard copy, which they dreaded keying in again. They heard about the scanner, had electronic copy back in an afternoon, asked how much we charged, and were told it was gratis to the Provost.

Episodes of this kind provided credibility to the project in the university at large and away from the desks of the growing number of humanists who found computers essential to their work. These were, predictably, often editors of journals or of literary and religious documents and monuments, or authors the first editions of whose writings were still presentable exclusively in typescript or print and who wanted electronic copy simply for ease of revision. Some were beginning to be required by their publishers, even in those days, to provide electronic copy for everything, including new editions of older works. As significant as these activities were for the project, near its heart and soul, the best consequence of the Kurzweil was the establishment in the eyes of many outsiders of the validity of the work humanistic computing could do. Some seemed gradually to begin to understand that when it came to the presentation and manipulation of the languages of the world by and through computers, we were acquiring expertise that others would overlook until they needed it in an emergency. It is important to note that this insight was never shared by the official computing establishment of the university, to whom our work was and remains an inexplicable anomaly.

68

1984

The outside world let itself be felt one more time at the end of this critical period where 1983 was folding into 1984. The conspicuous reality of the year was that desktop personal computers were fast rendering CALIS and its comfortable home in a mini/mainframe environment totally obsolete. The coincidence that CALICO was opening its face to the public made all the difference in the world. With that opening, the need revealed itself for a generally available CALIS-like authoring tool for the IBM PC compatible environment, not PC-PILOT, not "Private Tutor," but something reasonably feature-rich and very easy to learn and use for the teacher/author. Not only did CALICO provide in the shape of the annual symposia an open forum where such things could be learned, it also unlocked the door on government funding for all sorts of academic projects. Full-page advertisements grace the inside covers of the early issues of the CALICO Journal, inviting application for government and academy language and technology collaboratives. This was the hand of Minnie McNeal Kenny at work, and a powerful hand it was. In response to such an ad, the Duke project submitted a proposal in the Winter of 1983 suggesting the rewriting of CALIS in the 'C' programming language with an eye toward its portability to all common computing platforms of the day, but especially that of IBM PC compatibles. Attention to the problems of non-Roman character set representation was included in the proposal even though the opening provided by the EGA standard was still several years away. Our experience with the Victor 9000 permitted the project to speak with some authority on this issue, while not proposing Victor or any other single brand name as the solution.

The highest levels of the university supported the proposal, principally on account of its impact on international studies. The proposal was adorned with powerful and thoughtful letters of endorsement from the hand of Dr. Brodie and A. Kenneth Pye, Director of International Studies, later to be Chancellor at Duke and, later yet, President of Southern Methodist University in Dallas. The university development office lent the help of one of its most talented officers, Myrna Jackson, to the cause. This loan included rigorous editorial supervision, instruction and modelling for budgets (about which none of us in the project had the vaguest idea), getting approvals, observing technicalities, and hands-on help with collating, duplicating, and mailing. Finally, the gentle reminders, both here and there, the follow-ups, the proprieties and courtesies

69

attendant upon a site visit, all of these assured that everything that could be done by Duke institutionally to assure success was being done. The rest was up to the government.

The proposal rattled around government for six to eight months, when the gentle prodding of Myrna Jackson combined with a notice by Minnie Kenny that there was a proposal that needed looking at caused the document to fall into the hands of doctoral alumnus Stephen Cole, of the University of North Carolina at Chapel Hill, then a government linguist with a burning interest in ancient and less frequently taught languages. He joined a team headed by Ed Budraitis, contract manager for the National Cryptologic School (NCS), a Department of Defense training entity in which a great deal of language instruction takes place. Eventually they decided on a site visit to Duke to see just what was going on here. The team inspected the Kurzweil operation, saw H-P CALIS at work, liked the work of the Ibycus project best of all they were shown. The chief officers of the University received them cordially in interview.

It was only at one point, when the inspecting team met with interviewers from the official computing establishment at Duke, that a serious derailment was threatened. For reasons hard to fathom one of the interviewers started to invoke the abstruse vocabulary of one of the more theoretical fields of computing as a kind of a challenge or provocation. Steve Cole was up to the challenge and parried successfully, indeed more than successfully, with a razor response at the front end of cold steel knowledge and reasoning. It was a very tense moment. The mystery remains why the challenge was made in the first place. The odd behavior made it clear that reasonable cooperation from that quarter of the university was not to be expected. That unhappy news came as close to ruining the prospects of an agreement as anything that happened during the site visit.

1985

The prospects were, however, finally not ruined and beginning in FY (fiscal year) '85, the Duke Project began to receive generous funding first to write the port of H-P CALIS to the 'C' programming language and to accommodate CALIS to non-Roman orthographies. Jeff Gillette of the Duke Language Toolkit was brought in as chief

70

programmer, and others were brought in, at first part time, to help with newer tasks. Bob Gerstmyer, also a student of Religion, came on board with the job of automating as well as one could the cascade of new features which were, perforce, making CALTS, to be sure, more powerful but also harder to use. As the new PC version of CALIS began to respond to the needs of the language teachers and become thereby self-conscious of methodology for the first time, more and more features got added, some in the areas of answer processing and housekeeping, others in the area of conditional processing ("branching"), but most in the area of screen design.

"Can we scroll through texts larger than a screen? What can we do about skimming when it comes to reading comprehension? What about scanning? Can't we automate lexical chaining? Can't we automate CLOZE exercises? How about answer processing for language production? How about stressing recognition?" As each of these expectations began to be met, the initial simplicity of CALIS was sacrificed and the need for a simple interface grew more and more acute. The work of a Bob Gerstmyer came more and more in demand: how does one add complexity and yet conceal it to provide the continued illusion of simplicity? The result was the first generation of the AUTHOR program, by means of which a relatively straight forward markup scheme could transform a text into several kinds of CLOZE exercise or variations, such as a chaining exercise.

1986

These early years of government funding allowed a great deal of flexibility and the project started to support a greater array of activities. Onto the CALIS agenda came Amharic, the language of the ruling caste of Ethiopia, as a "worst case." The language has slightly different symbols for each vowel consonant combination, resulting in some 283 distinct characters. CALIS needed to be able to represent them all as well as the Roman alphabet and enough of the International Phonetic Alphabet (IPA) to describe the Ethiopic sounds. The first time we showed the screen alphabet to a native speaker and printed it out, we received the strangest reaction. The individual seemed deeply moved. It was clear that his relationship to the written language was far different from ours to our own. In his case the letters belonged somehow to the realm of the sacred. He

71

did not seem to think that technology was profaning them, rather that they were in some way sanctifying the technology.

To complicate matters, a search was undertaken for some reasonably effective audio device. A decision made in favor of a board manufactured by the Dialogic Corporation. The Amharic sound set was recorded digitally and presented in exercises to learn the character set. These were assembled by a remarkably talented Durham resident and native speaker, Tekola Fisseha. All the pieces were combined, with the Amharic character set on the screen with Roman and IPA transcriptions and the sounds of the language emanating from an attached speaker. This early instance of multimedia, still in the DOS world of IBM PC compatibles, was presented to the government authorities in Linthicum, Maryland, on January 2 8, 19 86, who seemed well pleased with what the taxpayer's dollar had bought. Alas, the day was that on which the Challenger went down and deeply grieved the local personnel, many of whom had worked at NASA or had good friends there.

In addition to Minnie Kenny, Steve Cole, and Jeff Knisbacher, who saw the Project through its first period of government support, a cast of other characters touched the workings of the Project over the years. These included Sally Schwarzkopf, Galen Clark, Bob Cullen, Carolyn Crooks, Pat Fisher, and Whitney Reed. To each of these names one or more anecdotes could be attached demonstrating the instructive but also colorful and adventurous character of the complex relations that constitute a serious project in its many facets and activities.

In the summer of 1986 the Project conducted one of that early sequence of CALICO workshops, participation in which seemed to determine a large fraction of those individuals who would remain actively involved in the field of CALL, the other venues being Brigham Young University, Middlebury College, and University of Michigan, before the tradition wore down. The neighborly generosity of the training division of the northern Telecom Corporation provided the project with an Ashton videotape interface. With this device and a version of CALIS tailored for video, it was possible to prove that some impressive multimedia materials could be developed on the fly, by an independent teacher with a bit of enthusiasm and without the support of a six or seven figure project. Gary Smith of the College of William and Mary, and Jeff and Anita Knisbacher, government linguists, were among the instructors and participants who

72

were exploring the range of video interfaces available at the time to put interactive video into the hands of the classroom teacher. It was again a question of trying to make inherent complexity seem easy and straight-forward.

The long promised largess of one of the industry leaders at last appeared on the books in 1986 in the shape of a three year contract worth, to begin with, $250,000. It was scheduled to provide a PC based, user-friendly interface for the work that humanists would be doing chiefly at remote mini/mainframe sites, such as concordancing, text analysis, and "corpus linguistics" (the term was not current locally in 1986, though the activity was). This arrangement brought some new technology to the project in the shape of PC XTs for users and the wonderful new PC ATs for developers. Even though the COLOE project was the designated agent for fulfilling the contract, all the operating funds were coopted by an office of the computing establishment and directed instead to an undergraduate-focused project to reinvent Borland's Sidekick™ for the student market. At its conclusion, the project was to supply a diskette that would be given to incoming students for quick installation on their PCs. The diskette would include a memory resident address book, telephone book, phone dialer, calculator, editor, and, most importantly, calendar and scheduler that would somehow take advantage of the university's registration mechanisms. This part of the project might have succeeded had the officer in question not decided to assume personal oversight and assign a student as deputy for day-to-day operations.

The COLOE project met its obligations by constructing a menu with automatic communications utilities by which a scholar could employ machines arrayed in the language laboratory and with menu choice instantly hook up to the central mainframe computing service of the local universities and perform a variety of tasks. These might be to summon H-P CALIS or its record keeping function to check on the work of one's students or to employ John Smith's ARRAS program or the Oxford Concordance program and to do statistical work on such texts as were available electronically at the time. This activity imagined that mainframes would continue to have a role to play in the routine computing of teachers and students, which in 1986 was still a reasonable assumption, though growing less so month by month.

73

The rest of the year 1986 brought with it a quantity of presentations around the country as institutions were struggling between the desire to get informed, the fear of being left behind, and the overpowering inertia of wishing to keep things as they are. Of these presentations the most gratifying was at the Ethiopian Studies Association, meeting at Georgetown on October 4, where the work we were doing in Amharic was appreciated mostly for the possibility that it might slow down the process by which their children — they felt somehow uniquely among immigrants —were losing the mother tongue. It was a fascinating opportunity to observe the culture and interaction of a community in exile and how technology was being absorbed to serve its needs.

It turns out that there is something about the combination of languages and technology that seems to seek out traditional obstacles and to take especial pleasure in blasting them away. The function of languages, as communication has, of course, a great deal to do with it, but the excitement generated by the technology also seeks propagation and hates barriers. The international dimension of the project was making itself felt not just among foreigners here at home, but also abroad. In 1986 CALIS was adopted on a national basis in the Netherlands and released in a. very handsome spring binder with documentation in Dutch and examples of exercises in English, French, and German. The Institute for Curriculum Development in Enschede undertook the initiative. They followed with a second edition in 1988. In 1989, the Danes undertook a comparable adoption and publication, the work done at Orfeus, a curricular counseling service in Arhus. The CALIS project at home took a cue from what the Europeans were doing and assembled a release of CALIS with full documentation, also in a ring binder and sample lessons in print and on diskette. In the long term, hard copy turned out to be too costly, and the same materials are now distributed in electronic form in the release of text based CALIS. But somehow, it is not the same as having the consolation of the printed page.

1987

Superficially, the year 1987 was one of transition, in which the work of the project was being disseminated by the means usual in academic life, talks and workshops (at Transylvania University, American University, UCLA, and CALICO Monterey).

74

Beneath the surface, text based CALIS was maturing into a finished drill and practice engine, in combination with the Duke Language Toolkit, for the languages of the world. The features being demanded now had much more to do with record-keeping and networking than with pedagogic function. "Intelligent" tutoring and multimedia were still incomplete enough to warrant investigation and new effort.

At the time, multimedia could be dazzling, but the technology was still clumsy, especially when video was involved. The multimedia tower-computer, videotape or videodisc player, internal or external interface, InstaVox (random access analogue sound recording device, chiefly for providing speech at instructional speeds), and monitor (sometimes two monitors, one for text, one for video)-was a Rube Goldberg device requiring million dollar expertise to run. Every strand of cable cluttering up the back of the tower begged for catastrophe. The reduction of this monstrosity took developments in two areas, storage and band-width. The multimedia station of the mid 1990s has largely solved those problems.

Perhaps the freshest breeze of 1987 blew across the Project from San Diego and the meeting of the First International IEEE Conference on Neural Networks. It was an astonishing event which attracted thousands of participants from industry, government, and university. Peter Batke noted wryly that humanities computing, which had been around in organized fashion for more than a decade, was able to attract about two hundred and fifty participants to its biennial meeting, while neural networks, which no one was entirely sure existed, was able to attract thousands at its very first. This meeting restored to professional respectability a metaphor for computing-the nervous system of living creatures-that had fallen out of scientific favor some eighteen years before. This restoration brought (back) into the arena a serious competitor for the rule based, deductive parsing systems that had dominated artificial intelligence for decades. Neural networks were not supposed to distribute data according to a priori rules but rather to extract patterns from data (Borchardt 1988).

Artificial Neural Networks (ANNS) added some zest to the work of the Project. Several experiments were undertaken to see whether these concepts could be applied to the realities of classroom instruction. One of the earlier attempts addresses the phenomenon of "bogus input," that is, the bad habit certain students had of in keying

75

nonsense in order to “give up," have the program reveal the right answer, and doing the exercise over again perfectly from their notes. An ANN was trained on hundreds of instances of genuine and bogus attempts, tweaked in favor of tolerance toward bogus input, so that it would never misconstrue a genuine attempt for a bogus one but might miss one or two out of three bogus attempts. The important outcome would be that sooner or later the network could identify a bogus attempt and warn the student: "CALIS knows what you're doing and doesn't like it." It was eventually decided that this strategy was perhaps too Big Brotherish, and so it was never implemented. We felt, however, that we had proven that a useful application could be found for ANNS, that if they could learn to distinguish between legitimate and bogus input, they could learn other important patterns and distinctions (Borchardt, Geyer-Schulz, Janko, Staddon, and Wang 1991).

CALICO member Sofus Simonsen, colleague in the German Department at North Carolina State University in Raleigh, had a student that was struggling with the gender of German nouns. That student devised a network which he trained to recognize the gender of a random half of the nouns in his textbook (McKee 1987). This it did very well, learning over 98% of what it had been taught. He then tested the network for what it had learned on the other half of the nouns, and it got almost two out of three correct. And the mistakes that it made were rather like those students would make (chiefly, overgeneralizing).

One disturbing result of this experiment was the "second-best guess" phenomenon. If the network's highest certainty level was wrong, its second highest level was almost always right, and it had near zero certainty that the third alternative was possible. In all but one or two cases (das Ende), the network consistently guessed right about which gender the noun was not. This experiment and its variations were shown over the years to many visitors. One of these was the brother of the President of the Czech Republic, Dr. Ivan Havel. He was equally disturbed by the "second best guess phenomenon" and tried to force the network to reveal its reasoning by inputting slight variations of such a word, one vowel off or one consonant off, etc. The network, not surprisingly for people familiar with them, revealed no consistent pattern to these stimuli. In frustration, Dr.

76

Havel input a Czech word. The network responded after some "thought" (processing): "I am near 0% certain this is masculine; I am near 0% certain this is feminine; I am near 0% certain this is neuter." In sum, the network recognized that the input string was nothing like anything it had been trained on, that is, that the string was not German. We are calling this "the Havel Result." It suggests that an Artificial Neural Network is capable of distinguishing one language from another, at the very least, negatively (Andrews and Borchardt 1993). Frankly, this capacity is not unlike recognizing bogus input and distinguishing between it and legitimate input.

In collaboration with colleagues in the Slavic Languages Department at Duke, the Project undertook an experiment to compare the performance between human informants and an ANN when it came to nouns of ambiguous gender in Russian. The experiment seemed to prove that formal marking, morphology, something the ANN could learn from, while systematic and not random, was by itself insufficient to determine the gender of ambiguous Russian nouns, that phonetics (stress), perceived origin, and semantics were at least equally influential in human beings making gender assignments (Andrews and Borchardt 1993).

For the specific work of the Project, the most important experiment was one that could be applied during the phonetic inputting of Chinese. Twenty-three word classes were distinguished, an ANN trained on several hundred pages of Chinese text, and a weight table produced which indicated for each of the word classes with what probability that part of speech or word class would follow. The network was correct in the first guess about two out of three times, in the second guess about eight out of ten, and in the third guess about ninety-nine out of a hundred. Considering that there were twenty other word classes to go, this result was considered quite satisfactory (Yuan, Kunst, and Borchardt 1994). Still on the Project's agenda is the improvement of the first guess result, perhaps by extending the window over the sentence from two words to three.6

1988-1991

A changing of the guard took place in 1988. Donald C. Mullen came on board as chief programmer, succeeding Jeff Gillette, with whom he had worked part time before. Albert Wolf took over Peter Batke's responsibilities. Mary Zaim tried her hand at neural

77

network programming. In due course, Stuart L. Dabbs came on to assist Don Mullen, and Nick Staddon succeeded to the neural network task. Ted Bebenek, the team's "fireman," network administrator, and Slavic linguist, stayed on, as did Rick Kunst, guiding genius of strange character sets and things Chinese.

The work of this team brought three innovations to the functioning of CALIS. The earliest was the application of the "edit distance algorithm" to CALIS in the form of . By this function, CALIS could automatically compare a student answer with an anticipated right answer and identify discretely six kinds of spelling mistakes: 1) errors in capitalization, 2) wrong letter, 3) missing letter, 4) extra letter, 5) erroneously duplicated letter, and 6) inverted letters. The implementation was the work of Stu Dabbs and, later, of Oleg Verevka, visiting from the Crimea. We remember August '91 vividly, watching with him the disintegration of the Soviet Union on SCOLA, and his wondering to what radically different universe he would one day be returning.

The second innovation was the conversion of CALIS to the MS-Windows environment, a massive undertaking, but necessary as much for the inevitable popularity of the new interface as for the promised standardization of device connections — precisely that which would soon render multimedia a reasonable undertaking.

The third and most far-reaching was the decision to rewrite everything in due course for the UNICODE environment. This would not actually take place until the next phase of the CALIS development, but it required an education of everyone involved in the Project. In short, UNICODE is the industry's attempt to respond to the diversity of the languages of the world, realizing that not everyone speaks English and that not every language is served by the Roman alphabet. This means in practice the evolution of a standard for the representation of characters on the screen and internally to any program, one that would advance from 7 and 8 bits, the present standard, allowing for 128 and 256 characters to 2 bytes, 16 bits, allowing for some 65,000 characters. The last 30,000 or so are reserved for Chinese and the first 30,000 or so for the remainder of the world's languages. It turns out that virtually every line of code is somehow affected by this shift, that the entirety of WinCALIS 1.0 would have to be rewritten. The role of Chinese was greatly to increase in demand on the time of the Project and its personnel.

78

Culpeper (and other) Visiting Scholars

The intervention of a Duke Vice President, Joel L. Fleishman, allowed the Project to apply for support from the Charles E. Culpeper Foundation of Stamford, Connecticut. The shape of that support was a three year program (I 990-1993) of visiting scholars from around the world. All in all it was twelve fellows: Jack Burston, Joel Goldfield, Jurai Hord6ek, Wolfgang Janko, Wu Jianguo, Istvan Kecskés, Karen Kossuth, Miriam Schkolnik, Reinhard Schulz, Preben Späth, Gé Stoks, and Ben Verlinden. Their occupations included university professor (6), secondary school teacher (2), governmental or semi-governmental curriculum developer (3), language laboratory director (1). Two came from the U.S., and one each from Australia, Austria, Belgium, China, (then still) Czechoslovakia, Denmark, Germany, Hungary, Israel, and the Netherlands. Student assistants for the Fellows included young people from the U.S., China, and the former Soviet Union. Almost half of the Fellows elected to extend their stays at their own expense or that of their governments. Additional visiting scholars followed in the wake of those supported by Culpeper funds: from Japan (Akira Tateno), China (Lu Zhenyun, Yuan Mei), Kuwait (Wafa Al Muzaiel), Argentina (Gabriele Bauer) and Spain (Ramon Piqué).

To say that the Project profited from their presence would be an understatement. Professor Tateno caused there to be a final debugging of the code of text based CALIS as he recomposed it for the Japanese market. Mine. Lu contributed to the formation of Chinese WinCALIS, and Mine. Yuan wrote the neural network whose results help disambiguate pinyin inputting in Chinese WinCALIS. Wafa edited Arabic WinCALIS. Mr. Piqué is helping to design the new WinCALIS interface on the twin bases of language pedagogy and the psychology of human/machine interaction. Of all the gratifying experiences associated with the many activities of the Project, the parade of visitations, brief and extended, tops the list for satisfaction and building long-term friendships.

1991-1995

The most recent phase of the project began with the migration of CALICO from Brigham Young University and the stewardship of Frank Otto to Duke. This necessitated a careful division of labor and a meticulous avoidance of conflict of

79

interest. The tightrope to be negotiated ran between the technical, logistical support provided by the Duke Project to CALICO, and CALICO's neutrality over against the Duke Project. Eleanor Johnson, Laura Rhodes, and Kerrie Hudzinski embody that careful separation. Since the same hard core CALICO membership comprising Duke and its competitors and rivals has returned annually for the symposium since CALICO has come to Duke, the division of labor seems to be working.

That quantum increase in the activities in the basement of Duke's Language Center was accompanied by another changing of the guard in the staff of the Project. Venkatakrishna Kuncham came in to oversee the editor, Mohsen Mahdavi-Hezaveh the WinCALIS author template, and Raviram Medapati record keeping. Satsuki Scoville came on board for logistics and training assistance. Kunst and Bebenek provided long-term continuity in addition to their normal duties.

The activities of these years were dominated by the rewriting of WinCALIS for the UNICODE environment. From an American or Western European standpoint dual language computing satisfies most practical day-to-day uses of computers, including computer assisted language learning. For those needs, the ASCII character set will probably suffice for the foreseeable future. Most second languages can make ASCII pose as something else, say Cyrillic or Greek or Hebrew. No uniform standards have ever been developed for this pose, and most actual solutions are unique and unportable. When the proliferation of unique and unportable solutions is no longer considered tolerable, UNICODE or some uniform standard like it, is going to have to be adopted. WinCALIS is the first authoring and presentation environment to have taken the leap. It was not easy, which is demonstrated by the slowness with which alternatives are undertaking the same revision. However, a sense of accomplishment attends the display of Arabic, Chinese, English, and Hebrew on the same screen, indeed on the same line in WinCALIS, especially since that sight is accompanied by the certain knowledge that an internal representation of these languages bides behind their display on the screen. It is that internal representation that lets the program perform all the manipulations of language necessary to help students move that next step in their learning. Adopting the UNICODE standard, in connection with "string externalization" (rendering all the English in the interface replaceable by other languages), makes it possible to use WinCALIS to teach Korean to the French using a French interface: "Press

80

space bar to continue" ends up "Appuyez sur la barre d'espacement ou cliquez ici."

To prove that WinCALIS really worked, CD-ROMs were commissioned in Persian, Russian, and Ukrainian. The Persian, designed by Mohsen Mahdevi, turned out to be a multimedia extravaganza with video icons, calligraphy, photography, newsreels, poetry, song, and much, much more. The contents, given the sensitivities of that part of the world, are not uncontroversial, but the form is spectacular. It was an important demonstration that the program, to be sure, in the hands of a master, could do practically anything that any multimedia authoring environment could do. The next challenge would be to make it easy for other hands to do the same thing.

The Agenda

That leaves the foreseeable future to contend with. It would seem that after ten years of development the requirements should be winding down. Nothing of the kind. On the docket for the immediate future is the expansion of the on-line WinCALIS Korean dictionary to 20,000 words and the representation of the Thai language, on the screen and behind it, for WinCALIS. The languages of South Asia observe comparable conventions of representation, and the hope is that the Thai solution will be portable with ease to the languages of the subcontinent.

Certain theoretic issues emerge again here, so far into the development of the program, as animation, broadly and strictly defined, rises in importance in the multimedia universe. OLE (Object Linking and Embedding) will be the core technology of any new version, with a built in possibility of animating text ("teletype," TTY), and importing animation programs from elsewhere and running them within WinCALIS itself Perhaps the most practical contribution of UNICODE will be the throwback capacity to produce hard copy in printouts of any editorial activity, putting back into the hands of the classroom teacher Gutenberg technology for the languages of the world.

81

The next pressing agenda item is the completion of the record keeping module, above all, the ability to generate tests automatically, secure them electronically, grade them, generate printouts for teacher and student alike to post-process, in short, to try to reverse the disastrous failure that H-P CALIS had back in the early '80s. The hope would be to succeed finally in producing that essential piece of educational software that no teacher would want to be without because it saves so much time and does such a good job. At its best, the record keeping would be able to keep track not only of students over the years, but also of the educational materials themselves, the exercises and the test questions, to learn over time what questions were good and what answers and, by learning from correlation, just what tested what. In due course the program would be able to distinguish between and produce: 1) aptitude and achievement testing, 2) entry and exit testing, 3) placement, proficiency, and performance testing,

WinCALIS will work nicely on the MS-Windows 95 platform, but it is a 16 bit product and Windows 95 is a 32 bit environment. The rewriting of WinCALIS into version 3.0 is the next large task envisioned on the horizon. This will not be merely a technical tour de force but an opportunity to put into place the many developments in pedagogy and theory that will have been made by then. One imagines a proper “collaborative learning" engine, where the entire program is ready from the outset to accommodate solo, team, cooperative, or competitive learning tasks. Perhaps an authoring environment can be made to take instant advantage of the World Wide Web or its successor, and be as easy to use. Perhaps the work of the artificial intelligence community and its natural language processing component will at last be so economical as to be used routinely for intelligent language processing for all purposes, including intelligent tutoring and speech recognition.

There is enough to do to keep us all very busy.

82

NOTES

1 Aus Unserer Zeit: Dichter des zwanzigsten Jahrhunderts (New York: W.W. Norton, 1956); Querschnitt: Dicher des zwanzigsten Jahrhunderts (New York: W. W. Norton, 1962).

2 Occasionally student response had to be gauged by what they did and not what they said: it happened that students would spend ten minutes denouncing the program to interviewers and promptly turn around and obsessively work their CALIS exercises for the subsequent forty-five.

3 While it incidentally and inadvertently supported the “Pig-Latin” hypothesis, which is often postulated by second-language learners during the early stages of their learning: “L2 is L1 perversely encoded.” See the “Lexical Analogue Hypothesis” is Theodore V. Higgs, “Some Pre-Methodological Considerations in Foreign Language Teaching,” The Modern Language Journal 63, 7 (1979), 335-42; and more recently S.K. Bland, J. Noblitt, et al. “The Naive Lexical Hypothesis: Evidence from Computer Assisted Language Learning,” The Modern Language Journal 74, 4 (December 1990), 440-450.

4 Sixth International Conference on Computers and the Humanities, Sarah K. Burton and Douglas D. Short, eds. (Rockville, MD: Computer Science Press, 1983), at which Richard A. Kunst was already processing Chinese characters (pp. 772-80), the editor of this volume was already evangelizing Artificial Intelligence (pp. 595-99), and the honoree of this volume was already disseminating good sense (pp. 588-94) about language learning and technology. The author of this article also presented CALIS to the participants but too late to appear in the transactions: “Drilling, Testing, and Monitoring—Duke's Computer Augmented Language Instructional System,” June 7, 1983.

5 Wallace, William Norris. The medical metaphor in selected works of Friedrich Schiller; a computer-assisted study. M.A. Thesis. Durham, NC: Duke University.

6 We plan to continue to use ANNs for this experiment even though a consensus seems to have formed against them in favor of statistical strategies (chiefly Hidden Markov Models [HMMs]) for most pattern recognition tasks, including speech (and language) recognition. See Bernstein 995, esp. pp. 26-7.

AUTHOR'S BIODATA

Frank L. Borchardt took his A.B. at Saint Peter's College, Jersey City, in 1960 and his M.A. (1962) and Ph.D. (1965) at the Johns Hopkins University. He has taught at Northwestern University, Queens College, City University of New York, and Duke University (since 1972), where he is Professor of German and Director of the Humanities Computing Facility. He has been involved with the CALIS project since about 1979 and has been executive director of CALICO since 1991. Nowadays he inclines to publish more about computer assisted language learning than about the Renaissance and the Reformation.