Dodge Evaluation Question 1

As this question involves numerous aspects of the course, a number of specific questions were used to investigate the day-to-day functioning of the course:

Are the objectives and requirements of the course clearly stated?

After each project, students were asked open-ended and likert-like scale questions about the clarity of the objectives and requirements (see Appendices C and D). Students in both sessions rated the objectives and requirements of both the telephone and solar units as being clearly stated and easily understood, but several students were unsure of the underlying objectives of the telephone unit and/or the goal of the solar unit.

The limited confusion on the part of the students is probably due to the way in which the course was introduced. For the first session, most of the information on the purpose of the course, including goals, objectives, and requirements, was contained in the first packet of materials. This packet was distributed on the first day of the class, with the expectation that students would read it all by the next morning. Little in the way of additional introductory information or activities was provided. While most students claimed to have read the entire packet, their excitement, nervousness, and busy schedule on the first day may have impacted their ability to comprehend all of the detailed information that was provided to them.

During the second session, more in-class, verbal introductory material was presented. This seemed to alleviate some student confusion with respect to the objectives and requirements of the telephone unit. For both sessions, students were slightly unfocused when they began the solar unit. Possible causes and suggestions for improvement are covered later in this evaluation report.

Are the handouts easy-to-read and timely/useful?

Telephone unit. Students felt that the handouts were generally easy-to-read, useful, and distributed at appropriate times. Many first session students thought that the copies of the actual, 19th-century patents, caveats, and notebooks were hard to read, but an attempt to provide clearer copies to second session students resulted in fewer complaints about the copying. The "Do-It-Yourself" handout, which contained instructions for building different types of transmitters and receivers, were frequently cited as being the most useful handout. The copies of the patents and caveats were often mentioned as the least useful handouts.

The fact that students did not find the patents, caveats, and notebooks to be either readable or useful is interesting. The teacher clearly instructed the students that they did not have to read the patents and caveats on the first day, and that they were there strictly as examples of a patent and caveat from the 1870s. Yet almost all of the students read the material by the next morning (and complained about the quality of the photocopies and the 19th-century English). With this in mind, second session students were told that they should not read the patents and caveats. The purpose of the material was reinforced at least twice on the first day. However, students still did not seem to understand the limited purpose of the materials.

Solar unit. Students also felt that the handouts for the solar unit were generally easy-to-read, useful, and distributed at appropriate times. There were no comments on the legibility of the handouts, but several first session students remarked that the A. C. Rich patent was difficult to understand. Students from both sessions did not find the handouts, specifically the patent and advertisement, to be applicable to their project. Again, the purpose of the patent and advertisement (i.e., as an example of a modern patent) were explained to the students when the packet was distributed. Students were also told that they didn't have to read the packet. Student comments about the readability and lack of applicability of the solar handouts declined in the second session students, indicating that the emphasis on explaining the purpose of the handout during the second session was effective.

Suggestions:

• The motivation of the students was underestimated -- they would probably have read War and Peace if it was included in the packets. To save some paper, perhaps each group should have access at a given time to the master copies of the patents, caveats, and notebooks. If simple follow-up questions (e.g., What characteristic of the patent did you find to be most interesting?) are provided on a worksheet or as a guided journal entry, the students will have less material that they feel compelled to read, and they will probably get more out of the materials.
• One student suggested that teachers provide 5 minutes for students to read handouts and 5 to ask questions about the handouts after they are distributed. While few of the handouts could be read in 5 minutes, providing 5 or 10 minutes to discuss the previous day's handouts at the start of each class would help determine student comprehension of the materials.

Do quality and quantity of materials meet student needs?

As will be the case with the first iteration of any new course, student use of materials is hard to predict. However, logistical problems were minor at best. Students were able to find what they needed, or the instructors obtained it for them with 24 hours. In the few, isolated cases in which students complained about lack of materials, the situation was either a result of their own actions (e.g., dropping the flashlight, blowing the multi-meter fuse, misplacing tools, playing with materials in ways not related to the course) or were unnecessary. For example, one student demanded specific sizes of wood panels and boards and could not be convinced that they were not necessary. However, after considerable effort was utilized to obtain the wood, the student did not use them for his group's project.

With respect to the solar unit, students requested more access to research materials, especially the university libraries that were within walking distance. In the telephone unit, a great deal of reference material was provided to the students -- they also were only modifying pre-existing designs. In the solar unit, students were provided with considerably less in-class material. Many of the groups also designed inventions that were not modifications of previous designs. The combination of these two things necessitated greater use of library resources.

Interestingly, students did not comment on surveys or during interviews on the use of internet-based resources in their inventing processes. However, some of them were overheard discussing the merits and advantages of the internet. Perhaps the limited access restricted the usefulness of on-line data retrieval. Many students also wanted to use computers to prepare their patents and caveats (both text and drawings) as they were working with their models. A majority of these students have access to computers for word processing at home and/or school, and the absence of computers in the classroom was a barrier to be overcome for may students.

Suggestions:

• Requiring lists of materials for the solar project was an excellent idea. It cut down on the extraneous requests for materials and helped to focus the students.
• If on-line resources are introduced, students should have greater access to these resources during the course. Many students seemed excited about the internet, but their enthusiasm quickly waned.
• Access to computers for word processing and drawing should be available in-class, during study sessions, and during free time--at least one computer per group. One student woke up at 6:00AM so that she could use a counselor's computer to type up her group's patent! these computers could be networked, thereby solving the problem of greater access to the internet.

Is enough time provided for students to work on projects during and outside of class?

Students generally felt that enough time was provided for each unit (both in and out of class), although they lean toward wanting more time. However, the complaints about a lack of time were strongest from certain members of the second session group -- ironically, due to a modified schedule and the lack of a guest speaker, there was more time available for students during the second session. For those students who were highly enthusiastic, highly challenged, and highly motivated, a year-long course would have been too short. Conversely, the few students who did not enjoy their experience in the course would have felt that one week was too long.

Suggestions:

• Add more activities on "invention" to the first day of the course. In the first session, the word invention was only spoken a handful of times. Modifications were made to increase the discussion of invention, but these instances accounted for a very small portion of the first class, and no activities involving invention or creativity took place. This may have added to the feeling of some students that they did not know what the course was about until the middle or end of the first week.
• Split the course -- each unit could be expanded into a two week course, or both units could be retained and expanded into a four week course. This would accomplish several things:
  • A two week unit may be easier to schedule for summer programs
  • Less overlap between the two units would occur (some students cited the overlap as a distraction)
  • Several of the students appeared exhausted or "burnt out" by the beginning of the third week, while their enthusiasm and motivation was fairly constant through the middle of the second week.
  • More time would be available at the beginning of the course for students to research/be provided additional background information on the invention of the telephone.
  • More time would also be available for small group and whole class discussions on the handouts and group processes (i.e., the promotion of reflection), internet instruction, and access to computer and library resources.

Although some scheduling issues are mentioned above, some "nuts-and-bolts" aspects of the course need to be further analyzed. Most of the following suggestions were provided to program staff during the first session of the course, and substantial modifications were made to some activities during the second session.

Suggestions:

• In several instances, the underlying structure and purpose of specific parts of the course were not told directly to the students. For example, the purpose of some of the video segments was not clearly transmitted to students, who often were bored or confused about why they were watching certain things. When the purpose of each video was explained with greater clarity during the second session, students appeared to show a greater appreciation for the content of the videos. The above-mentioned example of verbally introducing the requirements and objectives of the course also applies here.
• Alternative methods for separating students into groups may be more useful than the learning style and hemispheric indicators that were used in both sessions. First, recent research has questioned the validity of similar instruments (e.g., Pittenger, 1994). Second, the amount of time that was spent working with these instruments did not result in the positive outcomes that one would expect. Some students immediately labeled each other as "a science person" or "a word person," and when problems in group interaction arose, the results of the instruments were immediately used as an excuse for why the students could not get along. Less time was spent on the instruments during the second session, and the correct interpretation of their results was stressed, resulting in less confusion on the part of the students. However, problems were still evident (see the case studies in Appendix E). An alternative may be to use Multiple Intelligences (MI) theory to explain differences in learning and problem-solving style (Gardner, 1983, 1993). Gardner is not averse to this perspective (Walters & Gardner, 1986), and MI theory has been applied to educational settings in this manner (Armstrong, 1994). If explained correctly, students will realize that there are different ways to solve the same problem. For example, a person with logical-mathematical intelligence may solve a scientific problem differently than a person with kinesthetic or spatial intelligence, but neither person is restricted to a certain career or field because of their preferred problem-solving style. Students who learn in a spatial manner would benefit from a hands-on activity, while a verbal-linguistic student may prefer to learn material through a discussion/seminar format. This area (MI theory) is relatively new to education, and applying it in the manner described above would expand our knowledge of MI theory's educational implications.
• Several students "dropped out" of their group's solar project. Although this was much more pronounced in the four person groups, it also occured in the smaller groups. While the structure of the unit is not the only cause, the solar project allowed almost any type of invention. Students argued for their idea, and in most cases, the students whose ideas were not chosen quickly lost their motivation. To the students' credit, most of them rebounded and contributed to the final design and product, but several did not. In a more restricted scenario, the entire project would not be argued over, just specific aspects of the group's approach, resulting in greater ownership for everyone involved (like in the telephone unit).
• If a purpose of the course is to expose high school students to invention (i.e., applied creativity), the solar unit does not promote this in its design. Little of what the students designed was applied in nature, and students did not reflect on what they did or why they did it. This is often a characteristic of many hands-on science programs -- in the end, they are merely "building for building's sake," with little intellectual benefit for the students. Interestingly, the one group that chose to use the optional scenario/simulation provided in the solar energy handout is the only group that dealt with major ethical issues. While this could be due to particular students in that group, the scenario may have also influenced the applied, reflective perspective. A scenario, or a number of different scenarios, should be provided for students.
• If a scenario similar to that used in the telephone unit was employed in the solar energy case, greater continuity would exist with respect to the group interaction experiences of the students. For example, several students who were disenchanted with their group were told that the second project would be completely different, so they might enjoy it (and corresponding group interactions) more than they had during the telephone unit. However, the students may have benefitted more from experiencing a second unit in which constraints still existed, yet the content was completely different. In other words, students would benefit from seeing that different domains may necessitate different processes, both individual and group, even if the scenarios are similar. That lesson, which educators find very difficult to instill in students, was lost when the students moved to a second, open-ended unit.