Discussion Strategies Huddle

Fall 2020

Research Space

TAs need to be aware that there are different levels of teaching (e.g., through grading, mentoring, facilitating discussion, lab instruction) and they need to be prepared for those contexts. Opportunity to develop standards and training materials for TAs to increase teaching quality at ISU.

Needs Analysis

The project team conducted individual interviews with five new and senior TAs at ISU. TAs were selected based on experience. Questions were based on prior and current teaching experience, teaching pedagogies, technological skills, and support TAs receive from the institution.

Solution

Based on interview responses, we found that TAs need help with learner engagement strategies.

Our instructional design project followed the action mapping instructional design model by Cathy Moore (Moore, 2014). The action mapping model was created for workplace training but has the potential to be adapted for our purpose, which is to train TAs.

Credits: Seçil Akıncı, Ananda Muhammad, Hannah Dong

The design model by Cathy Moore itself consists of four components. Figure 1 illustrates the interaction between each component. In the model of the model is what Moore calls the measurable goal which is represented by a dart board. This goal is characterized by the term increase or decrease. One example of a measurable goal that Moore provides is ‘Reduce infections’. For our instructional purpose, our measurable goal is not specifically characterized by the term increase or decrease, but it is hoped that if workshop participants have not implemented engaging discussion strategies, they would implement at least one strategy that was introduced in the workshop. The second component of the model is the observed behavior that will help reach the measurable goal, represented by a green triangle with an exclamation mark. Within our context, observed behavior will be that of what we can see during the workshop. This is also tied to our four learning objectives, which is why we have four triangles within our adapted model. The third component is what comes before the observed behavior, namely the practice activity that can lead to the intended observed behavior which is represented by the orange hand. Finally, the fourth component is the information needed for training participants to complete the practice activity, which is represented by the blue information (i) icons. Moore emphasizes that this should only include essential information for one to complete the practice activity successfully.

Figure 1. Action mapping model adapted for the Course Design Institute (CDI) workshop

Our reason for choosing this model is threefold. First, the model is tailored towards in-person training events, which fits our project that will be presented in the form of a CDI workshop. Second, the term action mapping refers to the goal of changing what people do and not merrily what they know. This perfectly fits our short and long-term instruction goals, which motivates TAs to plan and implement engaging discussion strategies within their particular teaching contexts. Finally, one of the goals of action mapping is to aid instructional designers in creating practice activities that are based on participants’ realities, not just to deliver an informational presentation.

Learning Objectives

By the end of the workshop, teaching assistants will be able to:

Explore the use of word clouds, role-playing, and student-led discussion as discussion facilitation strategies.

Compare the benefits and drawbacks of the three discussion facilitation strategies.

Evaluate the level of engagement in the discussion facilitation strategies.

Create an appropriate discussion strategy for your teaching context.

Final Instructional Materials Live Presentation

Instructional Decisions/Features of the Design

One instructional design decision was that one of the goals of the instructional design model is to change what people do and not merrily what they know. This perfectly fits our short and long-term instruction goals, which motivates TAs to plan and implement engaging discussion strategies within their particular teaching contexts. That was why we decided to have a self-reflection in the workbook for students to assess themselves after implementing the discussion facilitation strategies after the workshop. Ideally, we also would like to conduct confirmative evaluation after 3-6 months to observe TAs while teaching to see whether they successfully implement the strategies and to give them feedback.

Another instructional design decision that we made was based on the student engagement during a learning activity framework as conceptualized by Reeve (2012). One of the components of student engagement was emotional engagement. We engage students emotionally through activities, like the icebreaker that allows them to tap into their previous experience related to discussions which will hopefully spark their curiosity and enthusiasm in learning about engaging discussion facilitation strategies, so that they could avoid implementing discussion activities that bore students. Peer-discussion, which we embed throughout the workshop, may also spark enthusiasm because participants get to hear others’ ideas that they may find useful for their own teaching contexts.

Finally, we decided to give workshop participants some time at the end of the workshop to create a discussion facilitation strategy implementation plan, because we wanted to elicit agentic engagement (Reeve, 2012). That is, we wanted participants to enrich their learning activity by actively thinking about how they might go a step further by implementing what they have learned in the workshop. Their planning is aided by the workbook that we shared with them which has a specific place for them to write their lesson plan.

Evaluation of the Instructional Materials

Our plan to evaluate our design’s quality will be by working closely with the TAs interviewed for the needs analysis, instructional designers and program coordinators at CELT, and course coordinators in departments. We will first employ connoisseur-based formative assessment (Morrison et al., 2019) during the instructional design process to make sure instruction is effective and efficient. In other words, subject matter experts will be involved in the review of instructional materials, learning objectives, and assessments. Tessmer’s (2013) four stages of formative evaluation will be followed during the formative evaluation. Tessmer’s (2013) model consists of four cycles, including the involvement of 1) expert review, 2) one-to-one, 3) small group, and 4) field test. Our formative evaluation will involve instructional designers, learners, instructors, and subject matter experts. One or two learners/TAs that were interviewed and provided feedback for the needs analysis will act as our learners. Instructional designers at CELT and course coordinators (e.g. the coordinator for ENGL 180: Communication Skills for International Teaching Assistants) will be consulted as subject matter experts and instructional designers. As the instructor, the CELT program coordinator who will deliver the workshop will be included in the evaluation stage. In addition, our project members, as experienced TAs, will also contribute as subject matter experts, learners, and instructional designers. Given the time we are expected to finish this project, two of the stages in Tessmer”s (2013) model will be followed in this project as shown in Figure 2.

Figure 2. Formative evaluation steps that will be used (adopted from Tessmer, 2013).

First, experts’ comments and feedback will be asked for followed by one-to-one meetings with TAs. Expert review will be conducted with subject matter experts and instructional designers, namely, the program coordinator and instructional designers at CELT and course coordinators, who will help us identify errors and what changes need to be made. Learners will not be included in this phase of the evaluation. Some of the questions that will be considered during the expert review phase are shown in Figure 3. Dick et al. (2015) identifies five analyses that can be used by experts, including congruence analysis, content analysis, design analysis, feasibility analysis, and user analysis. These five analyses, along with the questions listed in the figure will be implemented during the expert review phase.

Figure 3. Questions and analyses to be used in expert review (adopted from Dick et al., 2015)

The following phase, one-to-one, will be conducted with graduate students who are currently teachers. One or two of the TAs whom we interviewed for the needs analysis will be presented with instructional materials. Since the workshop will be delivered online, these one-to-one sessions will also be conducted online and TAs will be asked to discuss the materials (e.g., appropriateness and feasibility) and share any comments and recommendations. Learner interaction will be recorded for later analysis. We are aware of the importance of the other two components in the model (i.e., small group and field test). Upon completing the first two steps in the model, steps three and four may be implemented depending on our time. Overall, formative evaluation will help us see what needs to be revised before finalizing instructional materials’ design.

In addition to formative evaluation, summative evaluation will be conducted after instructional materials are finalized. For this phase of the evaluation, completed materials will be shared with the aforementioned experts and TAs who participated in the one-to-one sessions and expert review and their feedback will be sought. This summative evaluation phase will attempt to answer the following question: “Did the workshop solve the problem?” After the workshop, we plan to ask TAs and the instructor to fill out a survey and/or conduct interviews with them to collect data and examine the training’s effectiveness. Another evaluation approach we wish to employ is confirmative (follow-up) evaluations (Morrison et al., 2019), that is, we would like to observe TAs while teaching to understand whether they transfer what they have learned into their teaching and intended outcomes have been achieved.

Content accuracy

We will evaluate the accuracy of the content. We will also consult instructional designers and a program coordinator at CELT to get their feedback and input. Our formative and summative evaluation will help us decide whether the content is accurate and addressed correctly.

Appropriateness of objectives

Whether objectives formulated adequately for TAs, activities, and the content of instruction will be identified through formative assessment. One-to-one sessions with TAs, conversations with experts, and their feedback will aid with the decision of appropriateness of learning objectives.

Instructional quality

Through the use of instructional materials, a few changes are expected to occur in learners. TAs who will attend the workshop will develop skills to facilitate discussions, and transferring their knowledge to students. Overall, we expect a cognitive and affective change in learners.

Visual Design

One-to-one sessions with TAs will provide valuable information about whether the instructional materials are visually appealing and consistent in terms of the use of color, font size, and unity. Expert feedback will be requested as well to evaluate the aesthetics of the design.

Product Quality

This will be evaluated by the expert group introduced above. The appropriateness of the materials will be identified through formative and summative assessments. Feedback from the TAs will also play a role at this stage.

Usability

Users will be asked to evaluate the ease, navigation, clarity, and efficiency of instructional materials and satisfactory level. Based on their feedback during recorded sessions and the project team’s observations, necessary changes will be made to the instructional materials.

Contributions of Each Team Member

Each team member worked responsibly and collaboratively throughout the project and contributed equally. As a team, we brainstormed learning objectives and instructional materials we wanted to include, finished our tasks on time, and met every Monday to give updates and discuss the next steps. We split up the sections before each assignment and each team member worked on their section, followed by a team meeting to give feedback and discuss the assignment/paper. Each one of us interviewed a TA for needs analysis and contributed to the analysis of interview responses. Below is the contribution of each team member.

Secil Akinci worked on the evaluation of the instructional materials and read through additional chapters in the textbook and checked out outside resources to finalize that section. She also worked on the context analysis and the role-play strategy (i.e. what is it, how is it used) and contributed to the collaborative discussions on Canvas such as the UDL activity. She also worked on the presentation deck file and the facilitator guide.

Ananda Muhammad worked on adapting Moore’s (2014) action mapping instructional model for the CDI workshop. She also worked on identifying the engaging discussion facilitation strategies (word cloud, role play, student-led discussions) to introduce in the workshop. In the write-up of the instructional design document, she primarily contributed to the components of instruction; sequencing of the instruction; and alignment between objectives, instructional activities, and assessments within the workshop. In designing the instructional materials, she contributed to creating the scenario for the role-play activity and helped develop the content of the facilitator guide. Finally, she prepared the PowerPoint presentation for the live presentation.

Hannah Dong worked on problem statements, significance of the issues, motivational strategies, Learning Objective II’s instructional activities, and assessment. She also checked the content of each project and helped with the final presentation.

Priyankaa Krishnan worked on creating the instructional materials such as the facilitator guide and workbook for the workshop. She equally contributed towards the research, instructional design and group based assignments. She worked on creating the personas for the workshop and word-cloud discussion strategy. Finally, she made sure that the visual language of all the workshop instructional materials were consistent.

References

  1. Dick, W., Carey, L., & Carey, J. (2015). The systematic design of instruction (8th ed.). USA: Pearson.

  2. Moore, C. (2014). Action mapping on one page. ACTION@WORK. https://blog.cathy-moore.com/online-learning-conference-anti-handout/

  3. Morrison, G. R., Ross, S. J., Morrison, J. R., & Kalman, H. K. (2019). Designing effective instruction. John Wiley & Sons.

  4. Reeve, J. (2012). A self-determination theory perspective on student engagement. In Handbook of research on student engagement (pp. 149-172). Springer, Boston, MA.

  5. Tessmer, M. (2013). Planning and conducting formative evaluations: Routledge.