Needs and Satisfaction Analysis Report for Online Modules in Auditing Course (Acctg. 421)
1 Introduction
Dr. Martha Doran (the instructor) has created and delivered two online modules for course Accounting 421 – Auditing (Acctg. 421-Auditing). She likes to know how well she has done to the work and how she could improve it. The instructor wants the student instructional consultant to help evaluating existed online modules and propose recommendation so that she can enhance the modules scheduled on one month later and continue with improvement for coming semester. The instructor is especially interested in how students could gain judgment skills through online learning. She also likes to include more simulations in her online module.
Accounting 421 contains three units of auditing and three units of advanced financial accounting. This project is only working on the auditing section. Acctg. 421 - Auditing is about what, why, and how on auditing financial report as a professional certified public accountant. The instructor has taught the course for many years. This year, she decides to make a change to course delivery format by transferring twenty percent of the face-to-face classroom-teaching course to web-based online format. She chose three out of fifteen modules from the course to go online. The main consideration for her choice was to give flexibility to students’ tight class schedule. With a long history of interesting in computer-assisted learning, the instructor has intrinsic motivation to try taking big strides. The positive feedback for previous two online modules from students also contributes to her motivation to continue exploring new learning environment.
The institution encourages its instructors to use more distance learning in order to free more classroom space. They have had infrastructure ready for use. All instructors have to do is design and deliver the online modules, up to fifty percent of a course, on a reliable learning management system, Blackboard.
2 Audience Analysis
Audiences of the classes are accountancy major undergraduate students who have taken some classes on advance financial accounting. 97% of the students are in their senior years. 55% students are under age twenty-two; 21% is between twenty-three and twenty-five, and 15% is 30 or above. There is about the same number of female and male in the class. They are on campus students with no or very few experiences in online learning. The class, 80 undergraduate students divided into two sections, meet twice a week for three hours each meeting. Before taking online modules, they have met each other in classroom for advanced accounting classes.
3 Vision
During the first interview, the instructor talked about her long-term goals, “to shift more of the conceptual, knowledge based activities to the individual on-line method and then use the in-person class time for more group activities such as role simulations, group cases, presentations by groups, etc.” That implied, the instructor intended to do most of the knowledge dissemination and knowledge based activities (‘remembering’, ‘understanding’, and some ‘applying’ and ‘analyzing’ in revised Bloom’s taxonomy (Wilson, 2006)) online and let role simulations, group case presentation (‘evaluating’, ‘creating’, and some ‘applying’ and ‘analyzing’ in revised Bloom’s taxonomy (Wilson, 2006)) go to in-person class. She knew there were other possible ways to design her online modules better than they were now, and she liked to transfer 50% of the classes online.
Since it was difficult for the instructor to describe what an ideal online module should look like during the interview, I presented three evaluation tools from Quality Matters, EduTools, and California State University, Chico, to the instructor. I asked her to choose one tool that, she thought, would fit in her case better and can be a good tools for evaluating existed online modules as well as a good guideline for analyzing and designing new modules. She chose the tool from California State University, Chico (the Chico tool). It is reasonable to assume that the exemplary performances described in the Chico tool represented what the instructor thought an ideal online module should look like. Although the Chico tool aims at an online course rather than few online modules of a blended course like Acctg. 421 – Auditing, the tool can still be a proper reference to the project because, the long term goal of the institution and the instructor is to transfer 50% of the classes online. Only people who have a clear direction in mind arrive the destination.
4 Current Situation
A needs and satisfaction survey with plenty open-ended questions was conducted. The instructor encourage student to take the survey by providing extra credit. The return ratio was high. Seventy-six out of eighty students (95%) have responded to the survey. The answers to open-ended questions provided valuable information for both satisfaction and needs on online modules.
Besides the survey, I interviewed the instructor (as the subject matter expert and the incumbent of the project) and teaching assistance (as the colleague of the project) via instant message and/or email to learn more about the challenges they have met as well as the intention and expectation behind the computer screen. In addition, I reviewed two sets of extant materials: 1) students assignments submitted to Digital Dropbox in Blackboard to gain some ideas about the learning results; 2) course content on Blackboard website and textbook website.
Following sections present information gained from the survey, interview and extant literature.
4.1 Course Introduction
63% students thought the Overview and introduction for each online module were clearly written. However, 48% (44% in second module) students said they needed to check back and forth between different sources (sections on Blackboard, textbook website, or textbook; and PowerPoint slides), “It took a little while to figure out how exactly everything was going to work.”; “Most of the "back and forth" was between the ACL Tutorial and ACL assignment.” Students’ suggestions were:
“to put all the sources in one place so that I won't have to scramble out difference sources”
“the instructions for the module itself could have been shorter so that I'm not having to scroll up and down as much”
“having an online class as the first class was not the most ideal”
“a document with what is expected (from students) would be helpful”
“for the ACL assignment it would be nice to have a little more in depth introduction”.
46% students thought the instructional goals are clear. 83% students like to know how to meet specific instructional objectives.
4.2 Course Design and Delivery
Since the online modules were built upon Blackboard, the functional consistency, navigation flow, and accessibility issues were addressed automatically and there were not much left for the instructor to do aesthetic design; the survey didn’t contain those questions.
Interaction
There were not opportunities provided for any kinds of interaction because the instructor did not plan to do so, as she mentioned in the interview. 74% students agree or strongly agree that interaction needs to be included as part of the modules, although several students (less than 6.5%) said that they could and they liked to work independently through materials alone; or they could just communicate with other classmates on campus.
Some suggestions about online interaction from students were:
Q&A via email or the discussion board
Group work online assignment
Watch and analyze certain video about a company and what they do in their business
Small homework problems can help inspiring critical thinking
4.3 Assessment and Evaluation
Ongoing multiple assessments were used to assess students’ knowledge and skills. Students’ scores were designed to be published on Blackboard website. 92% students agree that the assessments were aligned with the learning outcomes stated on syllabus. Over 60% students said, most of the assessments were instructor-lead or self-check. Only 26% students strongly agreed that the grading policy was clear enough to them.
Students liked to have “more information about the time required completing the modules when assigned” because “With the planned 3 hrs for the assignment, it took about 5-6 to complete.” They also suggested having “discussion afterwards about how students did on the assignment”. They hoped the grades would not “took a long time to be posted” and “more feedback given”. Since students used ACL software for many assignments, some expressed that they liked to have “Tutorial and exercises on a software application”.
4.4 Learner Support and Resources
Domain knowledge required for attending the online modules
30% students said that they were ready for taking an online module and for the content presented in the modules. 66% students were not aware of the prior knowledge required for the course, although they knew Acctg. 421 must be derived from prior accounting classes. However, 71% students said they do not need a recap on prerequisite knowledge while 29% students said, “It would be nice to go over some things . . . to get a refresher”; and “What basic areas of past classes are applicable to auditing”. They said, “It would be good to have summary points about where to find items and any specific concepts that will be covered”.
Technology
Quite a few students said they continue to have problems with the “ACL software”, “Rogers Company files”, and “Digital Dropbox”. 64% students needed tutorials or handouts or an Q&A for general questions or an introduction video on how “to load and open correctly” and “unzipping a file” for technologies mentioned in previous sentence; and “access, excel”, “Quickbook and tax software”.
Other technologies
“There could be a virtual online chat room where the students and the professor could communicate effectively during an online course”, quoted from the survey.
Resources
Around 65% students said they had extensive or enough course specific and supplementary resources.
5 Extant Literature
Students’ assignment completing ratio was good, 95%. As to the quality of assignments, since they were graded by effort, no record showed how well students have learned from online modules.
Reading materials included in the textbook, textbook website, and course website were ample. Textbook and its supportive website contain most, if not all, of the materials and resources needed for learning and teaching the course. Such as, instructional objectives, scenario-based case studies, quizzes, flashcards, presentation slides, and relevant links for auditing. The challenge is how to re-structure the materials to meet the characteristics of students and the instructor so that the learning and teaching processes would run smoothly and effectively. For current online modules, students expressed their confusion on trying to locate specific information among various materials.
The course website was constructed upon Blackboard. Students received instructor’s announcements, submitted assignments, downloaded additional handouts here. Current course website appeared to be a container of agenda for each online modules, guidelines for things to do during each online modules, and other administration messages. Presentation slides from the instructor were posted here sometimes. Almost all learning activities use the resources provided by textbook website.
6 Between the Vision and Current Situation
Comparing current situation (data gathered from survey, interview, and extant literature) to exemplary performances described in the Chico tool, I identified following major gaps.
Category | Ideal Performances | Opportunities to Improve |
Course overview/ introduction |
|
|
Course design & delivery |
|
|
Assessment & evaluation |
|
|
Learner support & resources |
|
|
7 Finding Causes for Gaps
Purpose for blended learning was not clear to all participants
From administrative point of view, the management of education could be a reason for transferring parts of the classes online. For instructors, the purposes for going online have to be more than “answering to the request of the department”. Such as “increasing the quality, consistency, and accessibility of training”; ”maximizing content sharing and reuse”; and ”increasing learning effectiveness and throughput by institutionalizing best practices” (Office for Domestic Preparedness, & National Domestic Preparedness Consortium, n.d.) What the instructor stated in the interview was also a good reason for go online: “To shift more of the conceptual, knowledge based activities to the individual on-line method and then use the in-person class time for more group activities such as role simulations, group cases, presentations by groups, etc.” However, it seems that the instructor’s view on purpose was shared and discussed with the teaching assistance and students. The consequence was, students questioned about why they need to learn online and teaching assistance did not like the new change on submitting assignments online so much. If students and teaching assistance were aware of the benefits that online learning could bring to them and how they were going to try new environment progressively, they might be able to endure the imperfection raised during the process to a promising learning environment. They even might help improving the temporary weakness and defects.
Strategies for blended learning were not defined
What is the right mix of online and in-person modules? What topics to select for online? When is the best timing for starting the online learning practice? Is every element either in-person or online, or students could make their own choices? (Fransen & Swager, 2007).When these questions were raised and thought over, the online module won’t be merely a substitute of classroom teaching for convenience purpose. What kind of interaction can be applied online? What tool can be use to check students’ online learning outcomes? More ideas about fostering students’ competencies for creation, evaluation, and analysis will be discussed and realized.
Format for online learning was vague
Were the existed two online modules designed for self-paced or collaborative learning? Different format has different mixes of content, activities, assessments, and feedback system. There is no single formula suits them both. If the format were defined clearly in advance, students would have known what to expect (on-time help, real-person interaction, returning time, etc.) from online modules; and the instructional designer would not have missed elements that should have included.
Instructional objectives and methods to meet the objectives were not specified
There are learning objectives stated on textbook website for each chapter of the textbook but the average number of objectives for each chapter was twelve, far more from usually will suggest. I assumed the instructor has her own tailored learning objectives for each module; however, her list of objectives was not announced publicly. Either too many objectives or no objectives could cause students’ anxiety and abated their confidence to success.
Interaction was not considered as part of the online modules
It is not likely that high-level thinking, such as judgment skills expected by the instructor, would take place via online learning when student-instructor and student-student interaction were missing.
Assessment management were not paid enough attention
One teaching assistance cannot bear the workload for grading 80 students’ assignments without strategy while maintaining the quality and providing timely, meaningful feedback. Assessment management might be a small issue but it could induce a nightmare.
The survey responses also showed that students were concerned about grading transparency in online assessment.
Student support system were not settled
Much dissatisfaction was results of students’ low readiness for online learning. Students were not prepared for learning in new online environment. New technologies confused them. New challenging discipline made them anxious. In conclusion, new online learners need a full-range student support system.
8 Recommendations for Solution System
8.1 Faculty development program
New online instructors need more than computer skills. A program, available in both collaborative and in self-paced format, and both online and in-person model, that covers pedagogy, design and development, technology, and evaluation for online learning will greatly help new instructors knowing the strength and weakness, the known and unknown, the barriers and the possible directions to breakthrough. The program should include,
Discussion forum for clarifying the purposes and defining strategies for going online, as well as making choice between self-paced and collaborative format for online modules.
Learning facilitation skills that are particularly connected to online learning. Such as online learning environment establishing, online group processing, online community constructing etc.
Tools and tips for online interaction and assessment.
Introduce to a wider range of educational technology, such as podcast, blog, wiki, bookmark manager, live conferencing, online community builder, etc.
A standing online community for continuing development on online teaching competencies.
8.2 Job aids for faculty
No outcomes of training can sustain long enough unless there is on going support. Job aids can serve as handy, time saving tools or an explicit learning agent for online instructors. Not only by providing checklists, rubrics, decision aids, flowcharts, work samples, FAQ or step-by-step guidance that create in house, institutions can also support their online instructors by reorganizing the immense free resource in the internet. Useful job aids can cover a wide range of topics:
Reminder for drafting instructional objectives
List of possible ways to measure student performances categorized by types of performances
Examples for online interaction categorized by instructional activities and/or purposes
Rubric template for grading popular online performances, such as, participation in online discussion, reflection essay writing, and individual performance in group works for peer review, etc.
Checklist or template for creating managerial protocol, such as, grouping a team work, file naming for all assignments, policy for administration, netiquette for online discussion or debate, etc.
Step-by-step guidance for technologies used.
8.3 Learning support system
Blended learning environment have particular needs for strong learning support system because they constantly switch between in-person and online learning environment. They often forget they are partially learning at distance; therefore, striving for the company or on-time support and involvement of classmates or instructors (Oliver, 2007). Learning support system helps to form a sense of belonging, scaffolds learning activities, provides learning aids to students below expectation and offers additional resources to students exceed expectation. Following learner supports are recommended for Acctg. 421- Auditing.
Tutorials for technologies used
The survey found common demand for technical help with ACL software, Digital Dropbox, and Earthware software. Video or animation tutorials will be more effective for guiding complicate procedures than textual tutorials.
Recap for relevant knowledge from past courses
Provide links to prerequisite knowledge so that students could refresh their past knowledge related to the course.
Sample essay for completing assignments
No matter how hard instructors try to delineate the required format for assignments, there will always be gaps between the information instructors try to deliver and the information received by students. Sample essays will speak for themselves and eliminate discrepancy of understanding. Moreover, they make learners feel that the standards are reachable.
Differentiated instruction for supporting different educational background, intelligences and learning preferences
A class with large student group like Acctg. 421- Auditing, students possesses different educational background. Some might have strong discipline knowledge; others might only learn the basics. Some might be technology savvies; others might have barely enough computer literacy. Some students are self-driven learner; others rely on external pressure to push them moving forward. Former studies on learning styles, multiple intelligences, and culture diversity in learning community all call for differentiated instruction design. As the instructor mentioned, the Auditing online modules are designed for both undergraduate and graduate students. These two groups of students have different learning skills, educational background, and age; therefore, it is necessary to consider applying differentiated instruction theory by:
Provide more than one topic with different level of complexity for discussion forum, group work, and essay writing. Students can choose a topic based on their own comfort level.
Offer more than one choice of assessments with different level of difficulty and allow students to perform their understanding in various formats.
9 Conclusion
“I like the on-line sessions. I wish there were more of them”.
“I like the online classes because you get to be hands on”.
“(I like online teaching), it forces them (students) to think about the sites we visit and give their ideas, which in class not everyone does this, since there are so many students in a class”.
Above feedbacks from students and the instructor prove that online learning is an approach worth to explore, especially when it’s blended with traditional classroom learning. Blended instruction can be a door to begin exploits the potential of technology in improving the quality of learning and teaching for traditional classroom instruction.
Instructors have recognized the efficiency and convenience of instructional technology for online learning; however, instructors might not know that, in blended learning, a significant amount of student learning is often achieved through online instruction, if
purpose and strategy for go online were clearly defined
course structure and how/where students allocate their time in mastery of the course content were changed
forms and management of assessments were redesigned
learning activities were scaffold by full-range support system.
Appendix
Needs and Satisfaction Survey for Online Module of Acctg. 421 – Auditing
Click on the image to view complete survey.
References
Wilson, L. O. (2006). Anderson and Krathwohl - beyond bloom. Leslie Owen Wilson's Curriculum Pages. Retrieved October 21, 2008, from http://www.uwsp.edu/education/lwilson/curric/newtaxonomy.htm.
Office for Domestic Preparedness, & National Domestic Preparedness Consortium (n.d.). ODP - Blended Learning Approach. . Office for Domestic Preparedness. Retrieved October 22, 2008, from http://www.ojp.usdoj.gov/odp/blendedlearning/odp.htm#1.1
Fransen, J., & Swager, P. (2007). Blended Learning; Choices and Strategy - Guidelines for effective interaction in learning practices (p. 18). Centre for eLearning, INHOLLAND University for Professional Education. Retrieved October 23, 2008, from http://www.blend-xl.eu/files/FransenJenSwagerPChoicesandStrategy.pdf
Oliver, R. (n.d.). Using Blended Learning Approach to Enhance Teaching and Learning Outcomes in Higher Education. Edith Cowan University. Retrieved October 23, 2008, from http://elrond.scam.ecu.edu.au/oliver/2005/iaup2.pdf
