The Perseus hypermedia corpus has been evaluated throughout its development over the past seven years. This report summarizes the FIPSE-sponsored evaluation of assignments, activities and methods developed during the three academic years from the Fall of 1993 to Spring of 1996. During this time, Perseus has been used in a variety of ways, including: for directed assignments where students follow instructor-created paths or worksheets; for open-ended assignments where students use Perseus to investigate themes or concepts and then write papers or create their own Perseus-paths; as a translation tool; as the basis for classics "labs" where groups investigate classical evidence; as a library resource and supplement to the course; as a lecture/demonstration tool during class sessions; as a correlate to museum activities or assignments; for online exams; and as part of a team-taught, online telecourse.
The many syllabi and assignments created during this three-year period as well as previous years' evaluation reports may be found at the Perseus web site: http://www.perseus.tufts.edu. Readers are strongly encouraged to consult those electronic documents. The key findings are summarized into four categories.
Perseus amplifies and augments teaching and learning. A common theme reiterated throughout the evaluation is that Perseus brings much new content to a course. This amplification takes two forms: more texts (including some not available for students in print forms) and more images and maps than most department slide and map libraries offer; and an integrated corpus that allows text-oriented courses to easily add image-based context and vice versa. Another type of amplification often noted is that content may be accessed more quickly and easily than physical versions in libraries (mechanical advantage).
Perseus was also found to augment courses by allowing instructors to introduce new activities that would have otherwise been impossible. The philological tools allowed instructors in class to illustrate points with word analyses or to visually and easily correlate geographic characteristics and textual passages. In addition, entirely new courses were created that integrated the varied and multiple resources in Perseus. More importantly, Perseus empowered new kinds of student learning such as sophisticated philological investigations by students who knew no Greek, visual investigations of themes, and new discoveries by students alone or as part of a class.
Perseus requires substantial physical infrastructure investment. At every site, hardware and network problems caused frustration for faculty and students, substantial economic and human resources were necessary to make Perseus available in classrooms and laboratories, and laboratory staff had to be trained to support faculty and student access. These challenges are faced by all educational institutions introducing technology into instruction and reflect the larger learning curve investments taking place in all disciplines at the close of the twentieth century. One effect apparent when Perseus was used through the WWW was the relative ease of use when compared to the HyperCard-based version--students did not have to learn a new system but used the mostly familiar interfaces of web browsers. One result that recurred over the years is that self-reports on the system interface and learning effects are highly correlated while demographics, computer experience, and frequency of Perseus use are not statistically correlated with learning effects.
Perseus demands new conceptual infrastructures for teaching and learning. Instructors must learn to teach with Perseus and students must learn how to learn with Perseus, and both these requirements involve substantial amounts of time. Many instructors noted the large investments in time required to create assignments and Perseus-augmented lectures. Likewise, some students complained about the amounts of time it took to learn to use and access the system and to find the information needed. Several instructors noted that students took longer to complete assignments than anticipated and that classroom use often took longer than planned since interesting alternatives or additional examples could easily be pursued with the system. Instructors should take into consideration that the novelty and amount of work an innovation demands may lower student course evaluation results during the early years of adoption. Several instructors noted that Perseus raised their levels of expectations about the scope of material accessible to students. Likewise, students at schools where Perseus was used in multiple courses came to expect that such resources would always be available for use in their courses.
A number of opportunities and challenges related to teaching emerged. The traditional dilemma of how best to mix open-ended and guided instructional activities is exacerbated by the many possible uses that Perseus offers. Likewise, how to best use class time and what content is displaced when Perseus-based content is introduced into a course or curriculum must be considered. Perseus allowed instructors to model how they do their own research and the risks and time required to model research should be considered. Instructors must learn to evaluate electronic assignments. One instructor noted that more extensive feedback was made possible by having assignments and his comments in electronic form since he could leverage all the advantages of word processing while grading. Certainly, instructors and administrators must understand that iterative planning and implementation cycles are required over years rather than weeks or semesters, and appropriate allowances, resources, and rewards must be available.
Opportunities and challenges related to learning were also varied. Students are certainly motivated by Perseus, especially by the images. In the final year of the project, students were observed to work harder and better when their assignments were put on the web--the persistence of the assignment beyond the end of the course and the "publication" of the work are likely explanatory factors in this regard. Some students reported being overloaded by the amount of content available in Perseus. Likewise, some students were overwhelmed in lectures that included many Perseus examples and multiple verbal themes. The learning curve necessary to use Perseus tended to be more problematic in large, general studies courses than in advanced courses for classics majors who tended to recognize the time it took to learn to use the tool as an investment to be amortized over multiple courses.
Perseus is bringing systemic changes to fields of classics. There are several indicators that Perseus is changing the way that classics is taught and studied. There are more than 50 courses included on the Perseus web site representing more than a dozen colleges and two dozen instructors. These courses use Perseus in a variety of ways and illustrate the penetration of Perseus into the classics curriculum internationally. In some sites discussed in this report, multiple instructors used Perseus for several of their courses. Several instructors noted that Perseus use has led students, faculty, and administrators to see classics as technologically "plugged-in" leading to more recognition and resources on campus. While it is too soon to generalize, the new courses created based on Perseus tend to integrate textual and visual materials and illustrate ways to break down barriers between distinct areas such as philology and art history.
There are other, broader indicators of Perseus' influence on the field of classics. New faculty position announcements list computer experience as requisites. The entire edition of Perseus 1.0 has sold out and the current Perseus web site receives more than 25,000 hits per day. Popular textbooks now include Perseus companion paths, and a Spin-off company: Classical Technology Systems provides training and support. The Perseus project continues to attract funding for expanded work, and workshops and papers related to Perseus are standard fare at professional conferences in the multiple fields of classics as well as education.
The focus of this report is on FIPSE test sites with additional evidence from other sites where Perseus was used. The FIPSE test sites are seven colleges/universities (Ball State University, Holy Cross College, University of Michigan, Rhodes College, Tufts University, Wesleyan University, and the University of Wisconsin) that agreed to participate in development and evaluation. Over each of the three years of FIPSE support, different sites were selected for focused evaluation. In the third year, Ball State, Holy Cross, Michigan, and Rhodes were studied in detail.
Data were collected in several ways, including:
This data was analyzed during the summer of 1996 and together with results from the first two years, serve as the basis for this report. The report is organized as follows. First, summary sections for each of the four 1995-6 FIPSE sites are given, second, brief summaries of experiences at other sites are given, next an analysis of all student questionnaires returned in 1995-6 is given, and finally, a summary of the Perseus Listserv and website is provided.
Instructors at BSU continued to use Perseus in multiple courses and in different ways in the 1995-6 academic year. Walter Moskalew continued to use Perseus for his CC 105 (Classical World) course where he used Perseus in the classroom for illustrating lectures and discussions and students completed two assignments that required access to Perseus in the laboratory. The first assignment guided students through a series of vase paintings to introduce concepts and vocabulary and to give students practice in visual investigation. In the second assignment, students were able to select one of four topics to investigate (kylixes, architecture, weapons, or furniture) and to use Perseus to gather information to write an interpretive paper. Moskalew also continued to used Perseus as a warehouse for raw materials that were used in Persuasion presentations during class lectures.
Bill Magrath continued to systematically build on his previous years' experience with Perseus in CC 305. In the past three years he had added more Perseus iconography and increased the weight of Perseus assignments in grading. He reported at the Evaluation meeting that student course evaluations dropped over these years although he believes he received good student papers as a result of the availability of Perseus resources. In 1995, he made a fundamental shift to decrease the grade attributable to Perseus and gave students a choice between an open-ended group project (as in past years) or a pre-fabricated project. All students chose the pre-fabricated project. He noted that there is an inverse relationship between what and how much students are asked to do and their ratings of the course. He also articulated a need to discover how to mix open-ended and strongly guided assignments. These issues are central to the development of conceptual infrastructures for teaching and learning with Perseus and other technologies (see Marchionini & Crane, 1994; 1993-4 and 1994-5 FIPSE Evaluation reports).
With respect to the FIPSE evaluation, Chris Shea was the primary Perseus user in 1995-6. She used Perseus in her section of CC 105 (Classical World) and CC 304 (The Ancient City--see http://www.perseus.tufts.edu/classes/CS.96s.html for an overview and rationale for this course). Her instructor reflections explicate the time investments professors make in adopting Perseus and suggest ways that Perseus and classics departments can support the instructional needs of other disciplines. Professor Shea also kept a journal of her experiences over the semester. The questionnaire analysis for these two classes illustrate two extremes in student ratings and suggest that instructors consider how Perseus is used in different courses.
CC 105 is a general studies course that attracts a wide variety of students, most of whom are not classics majors. The CC 105 class in the Spring 1996 semester was composed of mainly freshman and Professor Shea remarked that they had high levels of computer literacy. Students were given assignments to use Perseus to find specific facts or objects (the first assignment required students to find seven objects, e.g., "What is an aulos? Find an object which depicts it.) and to find information about topics or themes (a second assignment required students to find five objects and describe the process used; e.g., "Print out a map, find Thera and mark it on the map. How far is it from Thera to Knossos?"). In her journal, she noted that students took longer than expected to complete the assignments. The most significant event of the semester was the server crash in mid-semester that deeply frustrated students and the instructor. Of the 30 questionnaires turned by these students, only eight said that they would use Perseus again, even if it were not required. Students complained about difficulty of use and time required to use Perseus. Typical comments follow for the questionnaire item: What was the worst thing about using Perseus? "Took too much time to find information--not very user-friendly." "It was hard to work. Things I needed to look up, I went through about every entry and still could not find what I wanted." "Not being able to find a Perseus program that worked." "It's difficult to find your way around." "Get lost a lot; sort of hard to use." "Trying to get the server up." "Took up a lot of time. Frustrated with path to information." "System problems, inconsistencies between net and CD-ROM versions." "I pulled all of my hair out and got a migraine headache." The few students who responded to the "Best thing about using Perseus" question made comments about the usefulness of images. The overall frustration these students felt is perhaps best summarized by one students' response to the final question about how Perseus affected learning: "[It] made it harder and made me mad!" These results contrast sharply with those from the CC 304 course.
In the CC 304 course, participants were upper-level students from a variety of majors. Professor Shea regularly used Perseus in the classroom to introduce concepts and model scholarly inquiry (e.g., in an early session she used it to conduct a "tour" of ancient Athens and suggested students follow up by conducting tours on their own). In her reflections she eloquently noted: "On days devoted to this serendipitous straying I believe we as professors are teaching the students the second of our great skills: learning. By taking them through the process of inquiry, assembling and (perhaps more importantly) culling data, learning is being modeled by the campus's best practitioners of that craft, by the professors, the ultimate lifelong learners."
These students were asked to maintain and hand in journals documenting their progress through the course. Students were free to use Perseus as a resource in writing short essays and were given some guided assignments (in one of these assignments, students were to simulate a walk around the Agora and answer five questions, e.g., "Choose a building you might have had some business in some sunny morning in ancient Athens. Describe it. Why did you choose it?" In her journal, Professor Shea noted that these assignments took longer than anticipated and she planned to abbreviate them in the future. She also noted some variations to try in another course--illustrating the important element of iterative planning necessary to develop the conceptual infrastructure for using technology in education.
The term project was linked to a museum exhibit on campus and required students to choose an object from the exhibit and explain its cultural significance to the people who created or used it. The project took the form of a paper and a presentation that could include Perseus demonstrations. These presentations were video taped and thus students were involved in orchestrating a complex process that demands alternative forms of expression. Students were told that their presentations could be used as "labels" for the objects in the museum, with the intention that students would be motivated by the authentic nature of the curatorial learning experience. That students are motivated to strive for excellence due to the persistent nature of their product (as opposed to assignments that only aim to achieve a grade) is echoed in remarks from Tufts and Holy Cross where students' web projects remain active after the course is over.
Students in this course were also affected by the three-week server outage and a late semester outage. However, the total experience clearly overcame these physical infrastructure problems. Ten of the 13 students turning in questionnaires reported that they would use Perseus again even if it were not required. It is important to note that only one of the 13 was a classics major and four were history majors. Some of their comments about why they would use it again follow. History major: "Very enjoyable just to browse through. Excellent visuals. Nice excerpts and documentation." History major: "I actually think I will enjoy Perseus more when I can skip from topic to topic. Having to trudge through a list of sites of questions diminishes the enjoyment of using Perseus." Anthropology major: "To look up things within the context of the culture for comparison/argument/etc." Psychology major: "Good information unobtainable elsewhere."
These students also noted problems of access in response to the "worst thing about Perseus" question. For example: Industrial technology major: "Finding the time to get to the lab--it would have been easier to use if I could have accessed it at home." History major: "When the system was down and I couldn't get anything done." History major: "Confusion about how to find things." The impact of system performance was directly expressed in one student's journal: "Perseus is fun to use when everything goes as it should or as it was programmed to run, but if even one glitch appears, the whole experience is ruined." Another student journal entry reflects the impact of system problems on students' self-esteem: "Met with Dr. Shea. Perseus system worked like a charm. I am not stupid!"
The images and maps in Perseus typically draw praise. A history major summarized an impression many students expressed: "Visually--much easier to understand concepts when you can see a photograph of an object and the scale of buildings and sites."
That Perseus images, maps, and texts were viewed as one key component of the course is illustrated by comments on the last question about how Perseus affected learning. An industrial technology major noted on the questionnaire: "After reading the book for the class and listening to the lectures, the Perseus program filled in any gaps or explained some things better." Another student journal entry described an integrative outcome: "My knowledge of Athens has come to its zenith. I have just analyzed something in depth, that I knew little about before coming into this class. Everything that I have learned seems to be represented by this one temple. Wow, they [the Greeks] did a pretty good job."
Another outcome that echoes some of the discussion in Tom Martin's journals of past years is the vocational theme. One student describes in a journal entry how learning about computers will lead to his career. The final sentence of that entry is: "In this program the past is linked to the future."
In both classes, Professor Shea was surprised at how much time students took to complete the assignments. Likewise, instructors are not immune to the time demands of tools such as Perseus. Professor Shea discusses in her reflections the time investments for professors as they plan to use Perseus in their course. She also notes in her March 11 journal the time tradeoffs during class time. This entry describes a session in which she ran out of time for a planned demonstration. "This, incidentally, points up the biggest problem with using Perseus in class--unlike slides which are too low-tech and familiar to excite much attention, Perseus itself distracts students and professors from the task at hand. (`How `bout an aerial here? Cool. Do that again. Oh, this one is much better. Wait a sec. Oh, no, it's not this one, it's the next one..." Bell.)"
As the questionnaire analysis shows (see Questionnaire Analysis section) the overall ratings of these two classes were quite different. The CC 105 ratings are the lowest of all groups returning questionnaires on most items, whereas the CC 304 ratings were consistently in the positive half of all groups. The following interpretation may be informative to others planning to use Perseus. In general studies, introductory courses that attract a wide variety of students, it may be unwise to create assignments that demand that students invest substantial time in learning to use a tool specialized to that course. Students, after all, are highly conscious of time-grade tradeoffs, especially for non-major courses. This tool learning curve tradeoff is strengthened when the tool is useful for only a portion of the course content (e.g., only the Greek part of a Greek and Roman course like CC 105). For such settings, using Perseus as a classroom augmentation to illustrate lectures and discussions seems to be the minimal solution. This interpretation is consistent with the Michigan experience, where students were able to use Perseus materials for assignments but through the mostly familiar (at least on the Michigan campus) World-Wide Web; i.e., they did not have to learn to use a new tool. In courses with mainly majors or concentrators and when the course can take advantage of Perseus throughout the entire semester, the investment in learning to use the Perseus tools may be judged to be above the time cost. Additionally, these students may see the value in learning the tool for subsequent courses.
In summary, it seems apparent that Perseus is a part of the culture at Ball State. Perseus acts as a library and a visual gateway for several of the BSU professors and students who are majors accept Perseus as a common resource rather than something experimental or special. The Classics department has leveraged it to gain new resources and campus attention (Perseus and the department were featured in a campus promotional video and Moskalew, Magrath, and Shea were featured in the alumni magazine). Most importantly, the faculty at BSU who have used Perseus over multiple years are developing the conceptual infrastructure necessary to teach with electronic resources. They actively share their experiences with other faculty at conferences and continue to study the impact over the years.
For the fall 1995, Professor Smith used Perseus in a Pausanius course in which students read selections from all ten books in Perseus. Professor Smith admitted that teaching this course would have been "impossible without Perseus" since no good Pausanias texts are available. The most visible proof of this were the voluntary presentations given by the students to an open audience, including a FIPSE evaluator and at least two professors at Holy Cross. Relying only on their knowledge of Greek and the use of Perseus, each student developed an original thesis. For instance, one student used the English to Greek word search for "narrative" and "story" in order to determine how Pausanias viewed his own work. During the presentation, the Perseus text was projected onto the screen to better illustrate his point. Professor Tom Martin commented to this student that without the Perseus tool, he would not have been able to make these findings. After all the students had finished, Professor Martin commended them on the `amount of original scholarship...I hope you will take this for granted ...I hope you will feel the sense of empowerment." Professor Smith also lauded them for doing what only someone with years of experience could do without Perseus.
Professor Smith also used Perseus in the Pausanias course to include archaeological materials for contextualizing the Greek language students were learning. The students were required to compare Pausanias' descriptions of the monuments in the Agora to the visual images of the Agora in Perseus. They then explored why Pausanias mentioned some monuments, but ignored others.
In the fall, Professor Smith also used Perseus to augment his two sections (each had more than 30 students with no archaeology background) of Introduction to Greek archaeology. Like the Pausanias course, all the course information was on the web, with many links directly to Perseus. One assignment involved reading Pindar's 7th Olympian Ode and then designing a monument for the ode's victor. The students' final project involved "teams of two or three students [who] will publish catalog entries of an object in the Worcester Art Museum on the World Wide Web." These catalog descriptions are still available on the web, for parents and friends to read (http://perseus.holycross.edu/Courses/Intro_Greek_Arch/wamguide/). Professor Smith also made it clear to a FIPSE evaluator that the students would not have been able to do this assignment without Perseus.
At the evaluation meeting Professor Smith noted that Perseus raises expectations for scope of accessible materials by students and for better quality student work by professors. He commented that students liked using Perseus to study for quizzes. He also noted that Perseus is part of the classroom environment at Holy Cross now and students are less aware of it's uniqueness.
In the Spring semester, Professor Smith used Perseus in an Art and archaeology in the Archaic age course (http://perseus.holycross.edu/Courses/Archaic/Syllabus.html) and two sections of an Ancient Science course. He noted that many science majors were in the ancient science course and their experience with long labs for other courses prepared them well for Perseus lab activities. He had those students analyze terms in Herodotus and found that they discovered things about Herodotus that he himself did not know (Herodotus was the subject of his dissertation); moreover students said that they had never before thought much about language or history until using Perseus for these assignments.
At the Evaluation meeting and in his journal, Professor Smith reported that Perseus was used by six professors, making Holy Cross the most Perseus-intensive campus in the world. (See the Perseus and Holy Cross websites for detailed syllabi and sample assignments for the courses described above.) Professor Smith offered three possibilities to explain why Perseus is so prevalent at Holy Cross:
On a similar note, the Greek history professor and Professor Smith both noted that when "technologically-borderline" professors made a mistake with Perseus in class, not only were the students able to correct him, but they also had more respect for the technology as they could relate to his problems with it. Furthermore, Professor Smith commented that one of his FIPSE funded student assistants went on to get a job at the campus computer lab. His advice to new Perseus professors: "get your technologically border-line colleagues committed; get your students trained' prop up the colleagues with students behind the scene, and you may bring the rest of the student body along."
This same history professor also shared with Professor Smith that "Perseus made a notable difference in their [student] performance [on the exams]. Even a single weekend...with a system permitting them to check themselves reliably on morphology and definitions seems to have made an impact."
The University of Michigan has a large classics department that provides several courses that serve as core undergraduate courses. For the 1995-6 academic year, Professor David Potter used Perseus in his Spring semester course: Classical Civilization 375: War in Greek and Roman Civilization (see web site for syllabus and assignments http://www.umich.edu/~classics/cc375/). Perseus was incorporated as an resource for assignments on Herodotus and Thucydides. A web site for the class was created by Sebastian Heath to support the class and provide access to Perseus and other electronic resources. It is important to note that Perseus was delivered via this web site rather than as a stand-alone system. Thus, students were able to move easily among Perseus and other web-based resources (e.g., the Celts home page) using one familiar interface. Although there were some remnants of the physical infrastructure problem highlighted in all previous years, these were limited to waiting for workstations to become available and some system crashes. Instead of the unique Perseus interface, however, students (and faculty) accessed Perseus resources through the ubiquitous Netscape interface with which they typically had experience in other courses. Although this limited what was possible (the specialized Perseus browser and some of the atlas were not available), it made implementation much easier at all instructional levels (campus network and labs, instructor, and student). This was perhaps the most significant result to come from the Michigan experience.
A site visit to Michigan was conducted on April 4, 1996 during which time students in one discussion section (approximately 15 students) as well as the two teaching assistants and Professor Potter were interviewed. Several themes common to other sites arose at Michigan.
Students reported finding the maps helpful in providing context in assignment one (one of the objectives of the assignment was to do exactly that). One student said: "There were a lot of maps on the web pages too so that does help you, when Professor Potter does point out all the locations on the maps because otherwise there would be just words there on a page, you know if you say this guy moved his army from point A to point B you have no idea where they are unless you have some background information, unless you know what those maps look line and where those cities are located and they did have good pictures on there that showed what the area looked like."
Students noted that information overload was a concern. One student's expression summarizes one aspect of overload: "It's hard to use so much information...A lot of it is information that you totally cannot use, its kinda annoying to have that in front of you because you have to go through all of that, get around it, and sometimes it takes a half and hour just to get into the section that you want to get into." Another student noted: "Not all of this is going to be important, not all of this could possibly be on the final. There is no way to figure out, there is a lot." Other students pointed out that it depends on what is wanted from a course: an interesting learning experience or a grade. "You have to look at what you want to get out of the class also, the professor tells you all of the detailed information because he or she, they know it themselves. And its up to you really if you want to memorize all the little facts, it does help to draw connections and it does give you background information and I do think that it does help you understand the overall issue of what is going on. I suppose what you really need to know, as far as what is necessary, is what is the issue going on at any particular point in time. I do think that all the little pieces of information do help you draw conclusions and help you understand what is going on." This tension between students who are grade and outcome oriented versus those who are experience and process oriented is independent of Perseus but Perseus (and WWW resources even more so) expands the potential resource base dramatically.
One student related the information access and use issue to a highly satisfying experience using online information resources in a political science course. He noted that he found resources on the WWW that were not otherwise available. He said: "Other classes [inaudible] in terms of accessing information and writing papers, and we won't ever go back to simply using books and resources like that. Now the first place I go is the Lycos research."
Perseus assignments were the essential difference between this course and offerings in previous years. In the past, students were not given assignments that were to be turned in (a paper and exams were the basis for grades). Sebastian Heath stated that the assignments were designed to give students a context for the topics of the course and "To get students to interact with the text." Just as Professor Shea at BSU had underestimated how much time students would take to complete assignments, Heath noted "...we designed them to take under an hour and I think we might have been off in that estimation, especially for the first one (upon prompting, he clarified the assignments took longer than anticipated). Professor Potter said that he wanted to make Perseus more interactive: "I think that one of the things that really has to be worked on in interactivity--they click and stare--it's where you have to start, but the next area of development is click, stare, and answer the question you were asked." He described the assignments as follows. "There were assignments in which you get the computer to ask them a question, which makes them go back and check the text do that they go back and they look at things a second or a third time." He also noted the importance of having the syllabus and the assignments online together (see the course web site for the syllabus and prompting questions). Thus, the innovation in the Perseus assignments at Michigan was in finding ways to prompt students with questions right from the system rather than verbally or with paper handouts. On the 35 student questionnaires, of the 19 students who responded to the "worst thing about Perseus" question, six noted system problems, six said the Perseus experience was a waste of time, and three noted that the assignments were not applicable to the course topics. Some of these reactions may be attributed to the expectations students bring to general studies courses, especially one that in previous versions had only one assignment and exams. However, as with the general studies course at BSU, the time investment in learning to use a specialized tool like Perseus, even on the ubiquitous WWW, should be considered as instructors integrate digital resources into those courses.
During the interview, Professor Potter pointed out the need for commentaries in Perseus texts but questioned whether it would be cost-effective to produce new commentaries. "I don't like uncommented text anyway, I think it is too foreign for students." He noted: "One of the problems is that Perseus uses Loebs and students buy Penguins," and argued that instructors will continue to use the printed texts that are commented. These comments were made as part of a larger discussion about the transition from a print to a mixed (BOTH print and electronic) educational culture.
Another theme that arose in the interview was related to the notion of a "plugged in classics" expressed by Professor Martin at Holy Cross in previous years and by students at BSU this year. Potter summed up his expectations about the effects of Perseus by pointing out the effects on student perceptions about classics. "I don't think there is any of this stuff [from which] you can expect great mental and moral transformations. I think what you can expect is just they generally make the subject more interesting to people, make them think. I think one thing we can say there are 600 people on this campus who know that classics is well plugged into the electronic age, and don't think that it is some boring discipline locked up in some dusty library somewhere. I think if there is any contribution it is, above all importance, our classics is here, it's here and now."
The Michigan experience demonstrated that delivering Perseus on the web greatly reduced the number of system specific problems for both instructors and students. This is not to say that all problems were solved (note the student questionnaire responses), however Sebastian Heath was able to spend time developing assignments and working with students rather than trying to get HyperCard-based Perseus to run on the campus network. Likewise, students did not need to learn a new, specific system but could use the Netscape browser with which most were already familiar. The use of assignments that prompt students to explore represents another direction for future development to make Perseus more interactive. A substantial side-effect contribution of this project at Michigan was the development of Heath's classics resources on the web, (http://rome.classics.lsa.umich.edu/welcome.html) which has become a heavily used link on the Perseus site itself.
Professor Kenneth Morrell took a new approach to teaching a Greek history class in the Fall 1995. Rather than rely on modern and ancient historians for a look at Greek history, the students used Perseus in a hands on approach to using primary materials to learn history through artistic and social developments.
Perseus was fully integrated and was the primary source of historical information and course assignments. His objectives ranged from traditional goals:
to more contemporary goals:
The course was composed of 14 students from a variety of majors. Students reported taking the class because they thought it would be interesting or because they knew Professor Morrell was teaching it. They also noted that they knew that Professor Morrell uses Perseus in his courses.
The course syllabus, information and all assignments were posted on a course web site (http://198.78.27.19/GRS/Courses/GRS/GRS221.html). Students were expected to turn in all assignments electronically. They used Perseus, Wordperfect or the word processor and spreadsheet in Claris Works, and a jpeg utility. They often inserted images into their papers, which seemed to have caused the most technological difficulty in the class.
Professor Morrell created a series of "labs" which he explains in the course syllabus
"are modelled on labs in the natural sciences. In teams of two or three, you will work with an extensive collection of digital images and selected secondary sources to become familiar with the architecture, pottery, and sculpture of the periods under consideration." Though the intention had been for these to be large group projects, after the first few unsuccessful attempts, they were done either independently or in small groups.
There was no text book used in this course; instead Professor Morrell had the students read sections of the Historical Overview in Perseus. The primary text authors used in this course were available for purchase in the bookstore or students were able to use Perseus.
Professor Morrell also gave "traditional" (content-based) examinations. In keeping with the non-traditional approach, however, he gave these on-line. Students were given a time during which they needed to go to the lab and download the exam document; when they were finished, they left the completed exam in an electronic drop-box for Professor Morrell. Initially, Professor Morrell had intended to leave electronic voice notes correcting the exams, but due to technological limitations, he resorted to cutting and pasting notes into the papers. He also noted that with this system, as opposed to the conventional handwritten notes, he was able to give much more feedback to the students as he could type faster and more legibly than he could write in longhand. Additionally, this electronic commentary helped in paper revisions, since he could directly link the two papers and highlight what should have been corrected. A key innovation that deserves future study is the notion that handling assignments electronically offers new possibilities for increasing the amount of feedback and number of student-faculty iterations. This notion should be testing in light of time-benefits and the potential for depersonalization of interaction.
Professor Morrell "spent considerable time incorporating Perseus, primarily in reviewing materials and designing lab assignments." For each of the six labs, Professor Morrell put very detailed directions on the web. Each had an Introduction, Objectives, and Description of Assignment. Professor Morrell also supplied study guides for each exam on the web. They were comprised of an Overview and Sections, which provided sample images and passages for review.
Another challenge he faced was how to best use class time. Professor Morrell's intention had been to use the computer in class. Due to Apple's failure to fill his order on time, however, he was not able to do so. When the laptop did arrive, rather than disrupt the schedule the course had already established, he rarely used the computer in class. Several sessions were held in the Macintosh Language Lab so Professor Morrell could demonstrate the labs before the students worked on their own. During the class observed during the site visit, for example, Professor Morrell led the students through an in-class "mini-lab" with the computer. One student expressed frustration, however, that things never worked as smoothly when working alone as they did when Professor Morrell was doing them. This comment echoed those of students at Ball State, illustrating again the investments students must make in learning to use Perseus itself.
In his reflections statement, Professor Morrell explained that the most difficult part of the course was evaluation "because I had to devote so much energy into designing materials." He expressed an observation made by other professors teaching with non-traditional methods: "I have yet to develop a satisfactory way of evaluating laboratory assignments, particularly for teams."
Though Professor Morrell was "unsure about how Perseus affected learning in this particular course" he was "certain ... that nearly every student learned to do something on the computer that they had never done before." He was not surprised by how many questions the students had about the computers, and his resolve to devote time to computers and information-seeking skills is in concert with the plugged in classics experiences at Ball State, Holy Cross, and Michigan.
A relatively common complaint about Perseus from history teachers is the lack of information about the Bronze Age. Professor Morrell again ran into this problem, but because his class was web-dependent, they were able to access notes for Jeremy Rutter's Bronze Age course at Michigan. Furthermore, the students were encouraged to email Professor Rutter. Professor Morrell notes that this would have been possible, but more difficult if done without the network: "Although there are a number of books on the Bronze Age, I do not believe that these notes or similar summaries and views are available in printed sources, and although it is possible for students to communicate with authors and faculty members at other institutions, the ease and immediacy of the network make such interaction far more likely to take place than if we were to rely on more conventional channels of communication."
Perseus and the Internet also provided materials that would otherwise have been unavailable to students at a small, non-research focused college. Professor Morrell explains that "this may be the first course I have taught using Perseus that students could do a few things that could not have been done before because some of the resources we used are simply not available in printed sources." He goes on: "The point I should stress ... is access for students who do not have the range of resources in our modest library to support similar approaches using printed resources. This is particularly true for the work we did on vase paintings. ... Our slide collection is not comprehensive enough to even begin to do what we did."
During the site visit and at the Evaluation meeting in May, Professor Morrell expressed frustration with the course. One factor was the infrastructure problems caused by the delay in Apple's delivery of the equipment. He felt he "might have been able to make better use of Perseus" if he had the equipment in the classroom. On the other hand he did note that: "The course promoted independent thought and creativity."
Not only was this Professor Morrell's first time teaching a Greek history course that spanned such a wide time frame (Bronze Age to the end of the Hellenistic Period), he was also faced with the challenges of the content limitations of Perseus, as noted above. "Consequently, I [Professor Morrell] struggled with the normal problems associated with teaching a course for the first time as well as problems with he infrastructure and certain limitations in the database. I do, however, look forward to the next iteration because I will have a much better sense of the pacing and the types of projects to try."
Time factors are always an issue when deciding what to concentrate on in a course and what to gloss over. At the Evaluation meeting, Professor Morrell noted: "Everything takes longer than we expect." When teaching this course in the future, he would cut the number of labs in half, time them better with what is being taught in the class, and shrink the time the lab takes to complete. The labs were very comprehensive and time consuming for the students, especially after they stopped working in teams. This assignment time issue reflects those at Ball State and Michigan as well.
During site visit interviews and on the Perseus questionnaires, the students seemed to have a reaction to the course that was opposite of Professor Morrell's--they thought it was quite successful. Knowing Professor Morrell and his style before the course, many of these students interviewed expected a technology component. When one student saw the syllabus, however, her first reaction was that she would not have enough time to do everything, "but it really has been easier than I would have thought." Another student expressed a similar intimidation at the beginning of the course, but towards the end, felt: "more confident now. I mean, Professor Morrell's always there to help, you can call him at home if you were having problems."
As illustrated at Holy Cross this year, the more a professor seems "down to earth" and like a "real person" (i.e. if the professor shared the students' frustration with computer troubles) to the students, the more credibility they and the technology gets. During an interview, two students were discussing Professor Morrell's accessibility and patience with the students when teaching them how to use Perseus. One of these students said: "You can not catch the man in a bad mood. Except when he's having problems with his computer. Even then, he's like, whatever, can't come to the phone, whatever." Both students then agreed that his attitude towards their learning makes learning easier since "he doesn't ever get that superior thing." This student continued that Professor Morrell was more interested in the fact that they did learn the history and the technology than producing the papers on time: "As long as it gets done, you can take your time. He can't make you become an expert in a matter of days."
This realistic approach to learning had a great impact on the students, since they all went away from the class feeling satisfied with what they had accomplished and how they had learned Greek history. While they were all impressed with the amount and the diversity of the content in Perseus, many of them did also recognize that they had approached learning differently in this course than in most others they had taken. One student expressed that this "hands on type of learning ... made learning fun and interesting...It was really a lot easier to learn the material by being able to see it myself."
Another student credited Perseus with helping Classics "come alive." This student continued: "Perseus is a wonderful supplement which adds to the professor's instructions and class discussions. For example, when looking at the images for our architecture, vase, and sculpture labs, the ability to read about the specific dimensions, unique characteristics, and the importance behind the piece not only gave me a deeper appreciation of the art itself but also a deeper appreciation for the craftsmen and their ability and skill. Also, it helps [me] understand the work when I was able to look it up in the Greek."
A Biology major explained in her end of the semester questionnaire that "it was much more dynamic and exciting to learn about Greek history from the cultural perspectives I gained through my use of the Perseus program." She continued that though "the course was definitely more challenging as a result of Perseus, the way Professor Morrell integrated the use of the program into the course promoted independent thought and creativity."
A History and Greek & Roman Studies major who had used Perseus in other courses at Rhodes, also thought that with the use of Perseus in the course, Professor Morrell expected more from students. At the same time, however, this student did not "think his expectations are so high that it's impossible to do well and learn." He also credited the diverse content of Perseus with allowing him to "extend my inquiries beyond texts." He also added that "it has become a vital part of my studies into the world of ancient Greece."
In summary, the experience at Rhodes reinforces several themes apparent at other sites. First, Professor Morrell invested enormous amounts of time in preparing assignments and integrating Perseus into the course; likewise students invested more time than the instructor expected to complete the assignments. Second, Professor experienced physical infrastructure problems, waiting for a laptop computer to arrive for classroom use and having to change his plans as a result. Third, Perseus served as a library and resource that extended the boundaries of materials available on campus. There were several assignment innovations at Rhodes as well. First, as at Holy Cross, there was a concerted effort to treat the Perseus assignments like the labs that students take in science courses. That the large group labs were abandoned for small group and individual labs in fact demonstrates that they were indeed like science labs that are typically small group oriented. Second, students read texts online using Perseus. This is somewhat radical and the lack of complaints suggests that these students accepted reading online as part of the course approach. Third, getting exams online is a step in the direction of fully integrating technology into courses and communicates the significance of the technology to students. Fourth, the use of web-based resources at other sites (Bronze age material) and the instructor's encouragement to students to send email to the Michigan professor is another illustration of how technology breaks down classroom wall barriers and opens up new avenues of learning. Although the instructor was somewhat disappointed that he was not able to achieve all his expectations, students were highly positive about the course and the role Perseus played in it.
Professor Robert Allison integrated Perseus into his spring 1996 course, Anthropology/Religion 225 Gods, Heroes, Magic, and Mysteries: Religion in Ancient Greece. Students could choose to do several of the readings directly on Perseus rather than from paper texts and Perseus served as a key research resource for student papers and assignments (about 60% of students' grades were dependent on using Perseus). See the electronic syllabus that contains links to instructions for using Perseus at Bates and well as to Perseus and other web-based materials. (http://www.bates.edu/Faculty/Philosophy%20and%20Religion/rel_225/syl-225-w96.htl).
In his presentation and discussion at the Spring Evaluation meeting, Professor Allison made several observations. He said that Perseus was an invaluable library for the Bates community, especially given the good campus network that provides widespread access (of 45 students in the class, only one did not have a computer in his/her room). He said that students were frustrated with the amount of unproductive time they spent using Perseus. He noted that this was the second time Perseus was used in this course and there was an enrollment dropoff from the first to the second year and within the Spring 96 course (from 70 registered to 45 who actually took the course). In the first course, there were a variety of problems using Perseus and so for the Spring 1996 course, everything was done on the web and he was able to include more Perseus activities based on his first year experience. He also initiated a newsgroup for the class and reported spending an extraordinary amount of time managing the electronic communications and the ongoing class project using Perseus. He was pleased with the student evaluations for the course and noted that there was a wide range of usage intensity across the class. His plans for next year are to develop a Perseus proficiency credential and perhaps be more topical rather than chronological in the overall course structure.
Miami University of Ohio and Skidmore College
Professors Suzanne Bonefas at Miami and Michael Arnush at Skidmore conducted an
innovative experiment in the Spring 1996 semester by team-teaching a course
titled Democracy in Athens. Once or twice a week, the classes would meet on
their own and once a week they would meet together using Apple Quick time
tele-conferencing software and a speaker phone. That day each student would
also have access during class to a computer with the course web page and the
day's notes on it. Professor Arnush did note, however, that students tended to
be more engaged in the web page at which they were looking than the discussion
that was taking place between both classes; when the web page was just
projected, the discussion tended to be of a better quality. Professors
Bonefas and Arnush would usually each teach part of the class and then they
would have a group discussion.
In one activity, the Miami students acted as a collective Socrates and the Skidmore students as the Athenians. Professor Bonefas noted at the Evaluation meeting that her four students gained a lot from the critical mass of students at Skidmore. She also noted that three of the four students now have web-related jobs and the fourth has an internship at Wired Magazine--clearly another example of the plugged in classics theme discussed at other campuses.
Because there was one speaker phone at each location and up to 22 students at Skidmore (Miami only had four), it was sometimes difficult to hear each other and they spent much of the beginning of the semester getting adjusted to having to speak louder than usual and often to repeat themselves. The classes also shared a newsgroup and discussion list. At the Evaluation meeting, Professor Bonefas noted that one very bright but `shy' student blossomed on the newsgroup.
Both professors were not as satisfied with the course as they hoped. One reason they both cited for this was the difference in class size between the two schools. Because the Miami students were so out numbered, they usually worked together rather than taking advantage of the MOO (MOO stands for MUD object oriented where MUD stands for multiuser dungeon) to collaborate with the Skidmore students. Professor Arnush also noted the difficulty of developing and uploading the web pages each week. Having taught this course four times before, without computers and with slides, Professor Arnush also mentioned that there were much more successful class discussions previously and the images looked better than the projected slide images from the web. Although this experiment did not meet all the professors' expectations, it did provide an existence proof for intercampus courses that take advantage of Perseus content and network technology.
At the Evaluation meeting, Professor Bonefas noted that many professors were creating web sites for various courses and often included pointers to Perseus from their sites. She has done so in the enormously successful Diotima web site that she maintains with Ross Scaife. She also raised an interesting tradeoff between web-based and CD-ROM version of Perseus by pointing out that downloading images from Perseus requires substantial RAM (always in shortage given the requirements of new versions of Netscape) and thus the CD-ROM version could be an attractive alternative. In her other classes (see the teaching with Perseus web site for syllabi and materials for six of her courses) she includes a number of in-class lab activities where she models the use of electronic resources and communication facilities and students follow up by working in small groups. She noted that in-class modeling is essential before "turning students loose" in the computer lab. Example lab activities for the Greek Civilization course include: examining Greek homes to better understand the culture and building a Greek building. For the former site, one group built a Temple to Dionysos with links to Perseus (see http://miavx1.muohio.edu/~bonefas/gkciv.html for the actual student projects). Professor Bonefas' experience with Perseus exemplifies how classics professors with technological skills are reshaping the teaching of classics by drawing materials from a wide variety of new electronic resources (such as Perseus) and linking students together in new collaborative work settings.
Tufts
Perseus development is based at Tufts and Professor Greg Crane has long used
Perseus in a variety of ways. In the Spring 1996 semester, he and Lisa Cerrato
used Perseus in a Greek language class. Due to problems in the Mac labs, they
used the web-based version of Perseus to introduce new Greek words and
paradigms. Cerretto noted in her course reflections: "My goals were: first, to
enhance the presentation of the grammatical studies with the HTML pages, and
second, to introduce the students to the Perseus tools and encourage them to
use all aspects of Web Perseus right away." She noted that this approach took
an enormous amount of time early in the course and after the first two weeks
she stopped developing vocabulary webpages and "devoted more of my time to
traditional instruction. Two reasons were behind this: the difficulty I had in
using the computer in class [she had to physically bring computer to class],
and the lack of student interest in these materials." She noted that students
were interested in the Perseus corpus itself and continued to use Perseus as a
translation aid tool. She gave an illustration of how Perseus helped her
convince students of the importance of certain verb forms--she used the Perseus
word frequency tool to show how one verb form was used over 19,000 times across
all Perseus authors but another is used only 19 times. She also used Perseus
images to contextualize the reading students did. She noted: "I discovered
that whenever Perseus was running, the class took time to ask a variety of
questions. Most of these inquiries were not specifically related to our
readings;" Students used Perseus in interesting ways to make presentations
about Greek culture and were very positive about these activities--to the point
of encouraging professors in other courses to use Perseus. These uses were
clearly outside the typical scope of a Greek language course and Cerreto noted:
"I felt that the need to liven and enrich the study of the language far
outweighed the need to complete additional lessons. If I had not used Perseus,
I believe that the class most likely would have studied two, possibly three,
more lessons." This issue of expanding the scope of courses as a result of
Perseus, especially the breaking down of distinctions between linguistic and
visual course, will be debated by pedagogists for generations to come.
The Classical Mythology course taught by Professor Halpern in the Spring of 1996 used Perseus as a supplement to readings and discussions. Perseus was not used in the class discussions by the professor, although the teaching assistant did do demonstrations in some classes. A significant problem was setting up equipment in the classroom and many equipment problems in the Mac lab. This course served as a general studies course for non-majors and as was noted in the BSU and Michigan sections, students do not see value in learning specialized tools for such courses outside their major area. Just as learning to use a centrifuge or other tool in a general studies chemistry course is part of understanding the culture of chemistry, so Perseus and electronic technology is becoming a part of the culture of classics at some departments. The burden of such learning will inevitably affect student course evaluations, as it did at BSU and in this case.
University of Houston
Professor Dora Pozzi used Hypercard Perseus 1.0 for class presentations in a
large Mythology course in Fall 95 and Gender and Race in Ancient Greek Myth in
the Spring 96. In the Mythology course, she found that this diverse student
body "respond[ed] very well when encouraged to watch actively, `reading' the
art, imagining the experiences that occurred in the spaces shown, and relating
what they see to what they read."
Some upper-level students were required to write a paper using Perseus as well. While Professor Pozzi admits that "the quality of the papers they wrote varies considerably," she also recognizes that "without exception, these students would not have been able to find even a small part of the evidence they got from Perseus [in books]." One student agreed that he would use Perseus again "because I could find and obtain the info without walking in a big library and getting tired quickly, I was able to spend a lot more of my energy actually thinking and learning than I would have otherwise."
Illinois Wesleyan University
Professor Nancy Sultan used Perseus in her May term course Classical Greek Art
and Architecture. Students "were asked to work in teams of two to create a path
on a topic related to the course theme...They were asked to create 20-22 stops,
with full annotation and bibliography." Students then presented the paths on
the last day of class. Some even created web sites to correlate with their
paths.
Professor Sultan explains the benefit of this assignment: "the students...found themselves learning about topics they would never have dreamed of without cruising through the program. In order to find a topic for a path, for instance, I suggest that they peruse the historical overview and check out the indices for art and architecture. One group discovered Fountain Houses and really got into the architecture and use of these little buildings. They never would have come up with that as a topic on their own. In fact, I gave the students a list of about 30 topics to chose from for their paths, and most students struck out on their own, finding topics that weren't even covered adequately on the program (they discovered after having been titillated with an image or two). For example, a group became fascinated with the little bronzes, but once they decided to do a path there were not enough bronzes on the program to satisfy them. No matter, they created a website to take up the slack. It was very creative of them."
Virginia Tech
In Professor Terry Papillon's Fall 1995 course, the Classical Age of Greece,
the Historical Overview and Encyclopedia were integrated into the course
syllabus, yet there was very little mention of Perseus as something special in
the course notes.
The students were given a Perseus assignment during the beginning of the semester entitled Achilles Project on Perseus, which required the students to produce a 4-6 page paper addressing a specific set of questions. In the assignment notes, Professor Papillon explained that the"goals of this project are threefold: to get you more familiar with the Perseus system, to continue thinking about Achilles as a part of Greece's world view, and to think about the relationship between verbal and visual arts in Greece." At the Evaluation meeting, he admitted that this was an ambitious list of goals, but also pointed out that "we need to be economical with our time."
The assignment required students to compare the literary descriptions of Achilles in Pindar, Sophocles, and the Odyssey and visual depictions of him on vases. Professor Papillon then gave a 26-step instruction sheet on how to use Perseus for this assignment, yet students complained of technical and access problems on evaluations.
At the Spring Evaluation meeting, Professor Papillon highlighted that Perseus allowed "students to make connections among different kinds of evidence." One student commented that Perseus was "much easier than trying to look up the same information in books and encyclopedias." Many of the students recognized that Perseus helped them conceptualize the ancient Greek world better than if they had not had the tool and so used it to study for exams.
A Perseus questionnaire has been used in all years of the evaluation (see Appendix A). In addition to the participating sites, a posting to the Perseus List was made asking instructors to have their students complete the questionnaire at the end of their Perseus experience. Table 1 summarizes the questionnaires returned for the 1995-6 academic year.
| University | Course | Number of Questionnaires |
|---|---|---|
| Fall 1995 | ||
| Houston | Greek & Roman Myth | 11 |
| Virginia Tech | Classical Humanities | 18 |
| Tufts | Latin 1 | 20 |
| Tufts | Classics of Greece | 21 |
| Tufts | Advanced Greek | 10 |
| Total Fall 95 | 80 | |
| Spring 1996 | ||
| Rhodes | History of Greece | 7 |
| Houston | Greek & Roman Myth | 10 |
| Ball State | Intro to the Ancient World | 30 |
| Ball State | Classical Myth & Theory | 13 |
| Johnson CCC | 20 | |
| Michigan | Classical Civilization | 35 |
| Total Spring 1996 | 115 | |
| Total 1995-6 | 195 |
The questionnaire elicits three types of data: demographic information about the respondent, assessments about the Perseus interface, and assessments about learning effects due to Perseus use. Demographic data (variable names in the analyses that follow are in upper case) include major, age (AGE), class standing, types of computers used, frequency of computer use (COMPUSE), number of software applications used (COMPAPPS), frequency of Perseus use (PUSE), number of Perseus features used (PFEAT), and hours using Perseus (PHOURS). Interface data include ease of learning to use Perseus (LEARNEAS), ease of use after learning (USEASE), frequency of feeling lost (LOST), ease of recovery when lost (RECOVER). Learning effects data include likelihood that needed information was in Perseus (INFONEED), confidence about finding information in Perseus (CONFID), degree of difference between Perseus and traditional assignments (ASNDIFF), degree Perseus contributed to understanding content of course (PCONTRIB), amount of useful information in Perseus (INFOAMT), value of Perseus use time to learning (VALUE), and satisfaction about learning with Perseus (SATIS). Other open-ended questions are also included on the questionnaire.
As was the case in previous years, correlations between variables showed that demographic data is less strongly related to either learning effect ratings or to interface ratings than are learning effects and interface ratings. Table 2 reports the number of Pearson Correlation coefficients that were statistically reliable at the .05 or lower level for each of the variables (note there are a 16 coefficients for each variable). Correlation statistics may be found in Appendix B.
| Demographic Variables | |
|---|---|
| AGE | 1 |
| COMPAPPS | 5 |
| COMPUSE | 9 |
| PFEAT | 4 |
| PHOURS | 4 |
| PUSE | 8 |
| Interface Variables | |
| USEASE | 12 |
| LEARNEAS | 11 |
| LOST | 13 |
| RECOVER | 12 |
| Learning Effect Variables | |
| ASNDIFF | 4 |
| CONFID | 11 |
| INFOAMT | 12 |
| INFONEED | 12 |
| PCONTRIB | 11 |
| VALUE | 10 |
| SATIS | 8 |
It is interesting that age is not correlated with any variable except frequency of computer use (r=.14, p=.05), with older students using computers less frequently than younger students. More surprising are the few correlations between amount of Perseus use and the other variables. It is not surprising that the number of times Perseus is used (PUSE) is correlated with many of the learning effect variables (the negative values are due to positive ratings coded as lowest numeric values). The few correlations for number of Perseus features and number of hours using Perseus perhaps suggests that users quickly form impressions about Perseus and these impressions may be difficult to change. These results are consistent with data from previous years and reinforce the finding that interfaces are more strongly related to learning effects than are previous computer experience, age or degree of system use. This result suggests that the success of technological innovations are more dependent on good interfaces and the first impressions that users gain with systems than on user characteristics and experience levels. This puts substantial burden on innovators to deliver attractive and usable systems.
A one-way analysis of variance across the eleven sections of questionnaires was run for the following variables: ASNDIFF, COMPAPPS, COMPUSE, CONDFID, INFOAMT, INFONEED, LEARNEAS, LOST, PCONTRIB, PFEAT, PHOURS, PUSE, RECOVER, SATIS, USEASE, and VALUE (see Appendix C for the complete ANOVA tables). All the runs showed statistically reliable (p<.05) differences in means across the different class sections except for COMPUSE and PHOURS. Table 3 summarizes the number of times a specific class section differed from the other ten sections based on a Tukey-HSD post hoc analysis at alpha .05. Clearly, students in section 531 (Ball State Introduction to Classical World) and to a lesser extent students in section 121 (Tufts Classics of Greece) rated their Perseus experiences quite differently than other groups. Examination of the means shows that the BSU class consistently were more negative in rating their Perseus experience. The Tufts group rated their experience more negatively than other groups but reported using statistically reliably more Perseus features. As discussed in the BSU and Tufts sections, learning to use a specialized tool in a general studies course (both these courses were mainly composed on non-majors) will typically lead to lower course evaluations.
| Group | Number of Statistically Reliable Differences |
|---|---|
| 121 | 5 |
| 131 | 2 |
| 141 | 1 |
| 311 | 1 |
| 411 | 2 |
| 531 | 10 |
| 532 | 0 |
| 711 | 1 |
| 811 | 0 |
| 812 | 1 |
| 911 | 0 |
Perseus Discussion List
Between September 1, 1995 and June 15, 1996 the Perseus discussion list
generated 271 messages. Two-hundred forty-six of these (90.8 percent of the
total) originated in the United States. Of the remaining 25, seven originated
from the United Kingdom, seven from Canada, five from Germany, and one each
from Australia, Brazil, France, Italy, Japan and the Netherlands.
Two-hundred fifty-five (94.1 percent) of the 271 messages originated from educational institutions, all but 4 of them colleges or universities (Table 1).
| Message originator | Number | Percent |
|---|---|---|
| All colleges and universities | 251 | 92.6 |
| U.S. colleges and universities | 228 | 84.1 |
| Foreign colleges and universities | 23 | 8.5 |
| Secondary schools | 4 | 1.5 |
| U.S. secondary schools | 3 | 1.1 |
| Foreign secondary schools | 1 | 0.4 |
| All educational institutions | 255 | 94.1 |
| All sources | 271 | 100.0 |
The 271 messages sent during the period under consideration were originated by 83 individuals. As Table 5 shows, some of these originators proved more voluble than others: while 49 individuals contributed just one message to the discussion, 5 contributed ten or more messages. In fact, these 5 originated a total of 109 messages, or 40.2 percent of the total number.
| No. of messages | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | >=10 |
|---|---|---|---|---|---|---|---|---|---|---|
| No. of originators | 49 | 8 | 10 | 3 | 2 | 2 | 1 | 1 | 2 | 5 |
The subject line of the messages proved to be an unreliable guide to their contents, particularly where message replies were concerned. Consequently, the topics discussed had to be determined by examining all 271 messages in turn. Since 18 of these messages dealt with two topics and 2 messages with three topics, the total number of topic "treatments" totaled 291 instead of 271. These treatments have been arranged into ten topic groups as shown in Table 6.
| Topic Group | Treatments | Percent |
|---|---|---|
| Problems running Perseus, display glitches, or software bugs | 71 | 24.4 |
| Perseus accessibility, availability, or distribution format | 63 | 21.6 |
| Broader implications of Perseus | 41 | 14.1 |
| New Perseus features and using available ones | 22 | 7.6 |
| Perseus spinoff proposals | 21 | 7.2 |
| Perseus evaluation and feedback | 18 | 6.2 |
| Copyright and licensing issues | 14 | 4.8 |
| Informational questions and answers | 7 | 2.4 |
| Suggestions for improvement | 7 | 2.4 |
| Other | 27 | 9.3 |
| Total number of topic treatments | 291 | 100.0 |
Not surprisingly--given that this is the first time anything like Perseus has been put on the Web--the majority of these treatments had to do with problems running Perseus, with 33 of the 71 treatments in this topic group focusing on Greek-text display problems in Netscape, and another 13 on apparent bugs in the Perseus software itself.
Sixty-three treatments, or 21.6 percent of the total, addressed issues of Perseus availability, accessibility, or distribution format. Thirty-six of these involved the format in which Perseus will be, or should be, distributed commercially.
Forty-one treatments (14.1 percent) discussed topics related to the broader implications of Perseus not only for the future of classical studies online (9 treatments), but also for the affordability of education (12 treatments), for the future of classroom instruction (19 treatments), and for the willingness of college administrators to back new technological initiatives (1 treatment).
Twenty-two treatments (7.6 percent) involved either announcements of features new to the Web version of Perseus, or questions relating to the use of available features. No single topic in this group generated more than two treatments.
A number of treatments (21, or 7.2 percent) dealt with proposals to apply Perseus technology in other contexts. A proposal to create a "Roman Perseus" (or, alternatively, "Latin Romulus") project appeared to stimulate particular interest, generating 14 treatments.
Fourteen treatments (5.9 percent) dealt with copyright or licensing issues most notably the question of whether Perseus images could be used in non-Perseus Web home pages without attribution or without license (9 treatments).
Eighteen treatments (6.2 percent) provided explicit general evaluations of Perseus. Although no negative evaluations were received, two subscribers asked to have their names removed from the list. Seven treatments (2.4 percent) contained suggestions for improving Perseus by the addition of new features. An additional seven treatments dealt with informational questions.
Twenty-seven treatments (9.3 percent) were placed in the "other" category. Eight of these dealt with Perseus-related administrative matters--meeting announcements and the like--and the remainder with matters having nothing to do with Perseus.
Clearly, the Perseus list is used by the Perseus community to deal with implementation and logistical issues. The range of participants illustrate the depth of Perseus penetration in the classics field.
Perseus Web Site Logs
The Perseus web site "took off" dramatically in 1995-6. Appendix D provides
one glimpse of usage, giving summary data for accesses from different countries
and domains. For the period July 25, 1966 to August 11, 1996, the site
transmitted 3,034,294 files, an average of 8694 files per day. By the time
classes were in session in September 1996, Perseus was receiving between 20,000
and 30,000 accesses per day. It should be noted that access rates are based on
page accesses rather than file transmissions (i.e., the access per day rate
does not count image files on a page as separate hits). This extremely high
hit rate (e.g., the entire Library of Congress site exceeds 100,000 per day)
demonstrates the range and size of the Perseus community.
Marchionini, G., & Crane, G. (1994). Evaluating hypermedia and learning: Methods and results from the Perseus Project. ACM Transactions on Information Systems, 12(1), 5---34.
http://www.perseus.tufts.edu