MIME Home | research | nwacc

NWACC Grant Proposal

Outcome assessment of a mobile wireless technology-learning environment on undergraduate engineering education.

Two faculty members from the Department of Industrial and Manufacturing Engineering (Dr. David Porter and Dr. Toni Doolen) are leading several initiatives to incorporate mobile wireless technology into the classroom.  Currently, a total of 86 undergraduate students and 14 graduate students are engaged in curriculum activities that involve the utilization of mobile wireless technology -- primarily Personal Digital Assistants (PDA's), laptops, access points, and wireless networking hardware/software.  The equipment being used in these efforts was received as part of the 2001 HP Mobility Grant.  Since the receipt of this equipment in July 2001, several new opportunities for the application of mobile wireless technology have been identified and efforts are underway to identify resources and additional sources of support to develop and evaluate the impact of a mobile wireless technology-based learning environment on undergraduate education in the College of Engineering.  PDA’s are an excellent platform to use in an otherwise traditional classroom because they require little infrastructure investment.  In addition, PDA’s that have wireless capability have the potential to impact the undergraduate classroom setting in an even broader manner.  While integrating leading edge technology in the classroom is appealing from a variety of perspectives, it is critical that the impact of the technology in the classroom be well understood so that the technology is used in a manner that enhances the education of all students.  The proposed work seeks to develop and test a methodology to evaluate the impact of mobile and wireless PDA usage on undergraduate engineering education. 

 

Innovation

The availability of classrooms with personal computing resources is not as pervasive as was projected a decade ago.  The costs associated with obtaining and maintaining hardware and software and the costs associated with wiring computer classrooms have made it difficult if not impossible for many higher education institutions to provide computer-enabled classrooms across their campuses.  The mobile wireless learning environment being developed in the College of Engineering at Oregon State University offers an attractive alternative to the traditional, desktop-based, computer enabled classroom.  Using PDA’s rather than their more expensive laptop or PC counterparts eliminates many of the cost and space constraints.  Using mobile versions of these devices eliminates the need for expensive infrastructure investments required to provide connectivity.  The objective of this effort is to enable the use of computing resources in any classroom setting – thus moving from a paradigm of computer enabled classrooms, to a paradigm of a mobile, wireless technology-based learning environment!

 

Impact

In an effort to pilot test the use of this technology, PDA's have been integrated into some of the in-class learning experiences in the course, Engineering Orientation II: Introduction to Engineering Problem Solving and Computing (ENGR 112).  ENGR 112 is the second introductory engineering orientation course for engineering freshmen.  Due to increasing enrollments in the College of Engineering, 80 students are enrolled in the course in the current quarter (Winter 2002), with an anticipated enrollment of 80 additional students in the course next quarter (Spring 2002).  The students in this course are engineering majors or students from exploratory studies who are considering engineering as a possible degree option.  One of the objectives of this course is to expose students to a broad range of engineering problems and demonstrate the critical problem solving and computing skills needed to address these problems.  In the past, the course has relied on standard lectures supplemented with a two-hour computer lab to help students learn and apply off-the-shelf applications such as Microsoft Excel and Visual Basic to specific engineering problems.  For the winter quarter, the in class-learning environment has been modified by the introduction of PDA’s in the lecture portion of the course.  The PDA’s have been used when new functions and features of Microsoft Excel are introduced.  This preview of the application in a lecture environment, eases the transition from class to lab.  Early use of the devices has already generated a level of excitement and enthusiasm not typically demonstrated by students in this course. 

 

Current use of the PDA’s in the classroom have taken advantage of the applications currently available on the wireless devices to enhance the learning environment, specifically, Microsoft Pocket Excel.  This has generated significant interest in the students, especially at the freshmen level.  However, this provides limited flexibility in advanced courses, both at the undergraduate and graduate level, when students are typically required to develop more challenging applications that, for example, access and manipulate resources that are external to the mobile device (i.e., distributed databases).  Work will begin this summer to design and deploy Java/E-Services applications that will further enhance the current capabilities of the mobile devices and allow the devices to be used in a wide variety of classrooms.  These development efforts will focus on both common applications as well as course specific applications. 

 

Examples of some of the common applications include student sign-in and in-class quiz capability.  These types of applications are broadly applicable to classroom environments and can be easily customized for a wide variety of courses not only in the College of Engineering but also across OSU and could be “exported” to other universities.  Class specific applications will also be developed and will focus on taking advantage of having increased computing capability available to all students real-time in a traditional lecture setting.  As a result of the planned development of a wider range of applications on the PDA’s, preliminary plans are underway to incorporate the PDA’s in a total of ten courses from the departments of Industrial and Manufacturing Engineering (IME) and Electrical and Computer Engineering (ECE) over the next year with the potential to impact hundreds of students as summarized in Table 1. 

 

Table 1.  Courses targeted to use Mobile Wireless Technology and Applications

Course Name  
Department
Level
Enrollment
Engineering Orientation I
IME
Undergrad
150
Engineering Orientation II
IME
Undergrad
150
Engineering Orientation I
ECE
Undergrad
200
Engineering Orientation II
ECE
Undergrad
200
Wireless Networks
IME
Undergrad/Grad
30
Information Technologies
IME
Undergrad/Grad
40
Telecommunication Concepts for E-Commerce
IME
Undergrad/Grad
30
Bar Codes and Automatic Data Capture
IME
Undergrad/Grad
40
Mobile Systems Engineering
IME
Grad
40
Integrated Industrial Systems Engineering
IME
Grad
40
Supplier Engineering
IME
Grad
30
Engineering Economics
IME
Undergrad/Grad
300
Engineering Management
IME
Undergrad/Grad
30

                                                                                    

The donation of hardware from Hewlett-Packard has created the possibility of developing a mobile, wireless learning environment, but much development remains to be truly effective in our efforts.  To date, efforts have focused on creating the infrastructure to support the use of the devices in the classroom setting.  For example, there are a number of logistical issues associated with checking devices in and out to students as they arrive and leave the classroom.  In addition, procedures have been developed to setup and maintain the devices (e.g. battery charging, transporting devices, and file and application administration).  Three undergraduate teaching assistants have been hired and were funded by the OSU College of Engineering (COE) to work with Dr. Doolen to develop the infrastructure necessary to integrate the devices into ENGR 112. 

 

Outcome Assessment

Although it is very early in the pilot testing of integrating this technology in a traditional classroom setting, a number of issues must be studied to ensure that the development of a mobile, wireless technology-based learning environment creates desirable outcomes.  We are requesting the support of NWACC to help us move forward in assessing the need for and the impact of our efforts to create this environment. 

 

Specifically, we are requesting the support of NWACC to hire a graduate student to support this effort by collecting requirements information from instructors who are interested in incorporating the devices into their course development, by developing training materials for these instructors, and by working with Dr. Doolen and Dr. Porter to develop the methodology and tools necessary to evaluate the impact of these technologies on the educational environment.  While exposure alone to the mobile wireless technology is important in developing the skills of tomorrow’s engineering work force, we are extremely interested in the pedagogical impact of introducing this type of technology into undergraduate engineering education.  Therefore, several research questions need to be explored:

 

To what extent does the use of mobile technology in the classroom enhance learning?

How can mobile technology be utilized most effectively in the classroom?

What are the issues associated with using mobile technologies in the classroom?

Do different populations react/respond differently to the introduction of mobile technology in the classroom?

 

In an effort to begin to study these questions, Dr. Doolen has enrolled two students from the OSU Department of Science and Mathematics Education to develop a questionnaire to study these issues.  These students, working on a project for a quantitative research methods course are committed to work with Dr. Doolen until the end of Spring Term 2002.  Over the next 10 weeks, these students will work to create a pilot survey to assess changes in behaviors and attitudes resulting from the incorporation of mobile wireless technology and applications in a traditional classroom setting.  Preliminary data collection from ENGR 112 will be used to validate the questionnaire.  Much additional work is needed, however, to incorporate the preliminary data from ENGR 112, to develop an appropriate analysis strategy, and to begin to answer the four research questions described.  In addition, the current project will look only at ENGR 112, but clearly as the mobile, wireless learning environment is utilized by a broader range of instructors and incorporated into multiple courses, this type of study will be invaluable to instructors in understanding how to integrate technology in a way that has a positive impact on students and their education.  Dr. Doolen and Dr. Porter request the support of NWACC to fund a graduate student to support the assessment and data collection processes necessary to evaluate the impact of this technology in the classroom.

 

Accessibility

In the pilot introduction of the PDA’s in ENGR 112, usage issues have been minimal.  Most students quickly learn to use the devices with little specific instruction.  One area of concern, however, is the use of these devices by students with disabilities.  In particular, the PDA’s would be difficult if not impossible for students with visual impairments to use.  Students with certain learning disabilities may also find the use of PDA’s a barrier rather than an enhancement to learning, particularly as we work towards using PDA’s to take on certain administrative aspects of course deployment.  In an effort to understand the relationship between the use of this technology in traditional classroom settings and students with disabilities, preliminary discussions have occurred with members of OSU’s Technology Access Program.  The requested funding would also be used to assess what issues exist in using these devices in a classroom setting for students with disabilities.  This assessment and any follow-on work would be done in collaboration with the OSU Technology Access Program and the Office of Services for Students with Disabilities.

 

Leverage/Feasibility

The College of Engineering and Department of Industrial and Manufacturing Engineering have committed to provide matching support for this work.  For the proposed work, the Department of Industrial Engineering will use College of Engineering fellowship funds to provide an additional $1,777 in stipend and OPE costs.  External funding for the development of applications using the PDA’s have been requested from Hewlett-Packard Company and the College of Engineering.  This support will be used to develop both common and course specific applications for the PDA’s.  While the generous support of Hewlett-Packard and College of Engineering have provided the hardware and resources needed to develop infrastructure to create a mobile, wireless technology learning environment, the NWACC support is critical in that it will provide the support needed to solidify our efforts to develop a robust methodology to complete outcome assessment.  In addition, this support will be instrumental in providing the data needed to assure that the development of a mobile, wireless technology-learning environment enhances the learning experiences of all students at OSU. 

 

Proposal Plan and Funding

Deployment and assessment of the mobile, wireless technology-learning environment are critical to the overall success of this project.  The specific tasks associated with the successful deployment and assessment of this project are summarized below.  These tasks would be supported by the NWACC-funded graduate student under the guidance of Dr. Doolen and Dr. Porter.  Approximate hours have been estimated for each of the tasks based on the pilot work currently underway.

 

§         Development of training materials for faculty interested in using PDA’s (40 hours)

§         Identification of issues of accessibility in using PDA’s in the classroom (40 hours)

§         Questionnaire development for outcome assessment (40 hours)

§         Data collection from courses using PDA’s in Fall 2002, Winter 2003, and Spring 2003 (50 hours)

§         Data analysis (130 hours)

 

The details for the requested funds and specific matching funds from the College of Engineering are summarized below:

 

Proposed Funding for GRA Costs

Item

NWACC

Amount

OSU

Amount

GRA Stipend - 1 GRA - 9 month appt. - 0.20 FTE (includes health insurance)

$3,475

$1624

Other Personnel Expenses (OPE) (3%)

NA

$153

Tuition GRA

$6,525

NA

Total

$10,000

$1,777

 


Schedule

 


 

 

 

Develop web page (September 2002)

Development of instructor training materials (October 2002)

Questionnaire development (November 2002 - January 2003)

Post questionnaire and instructor training materials to web page (January 2003)

Identification of accessibility issues (January 2003 – February 2003)

Post summary of accessibility issues and recommendations to web page (March 2003)

Data collection (December 2002 – June 2003)

Data analysis (April 2003 – September 2003)

Post summary of outcome assessment results to web page (October 2003)

 

NWACC Website