Join Episode 45 of the Creative Work Hour podcast as Wai-Ling Fong shares her dissertation journey, exploring the long-term impact of online teaching training during COVID-19. Insightful feedback and inspiration await!

Greg
Hello and welcome back to another episode of the Creative Work Hour podcast. This is episode 45, can you believe it? Today we're in for a very special treat. You've heard us talk about tiny desk concerts, but this is the lead up, the practice session leading up to a tiny desk concert. And we're very fortunate to have Wei-Ling Fong, who is doing a work in progress. towards defending her dissertation for her PhD. So what she will do, she will give a presentation that she has so far, and then at the end Creative Work Hour members will give some feedback for her.

Greg
This is a special treat. Alessandra, do you have any thoughts before we begin?

Alessandra
I'm good. I'm anxious to hear that, for as long as we have known Wai Ling, she has been working on this PhD. So it's very exciting for all of us. Wai Ling, I hand it over to you.

Wai Ling
Thank you. Good morning and good afternoon to everyone here. Thank you for being here to support my very first rough presentation of my dissertation. So March 2025 marks five years since the start of the COVID-19 pandemic. Around this time, five years ago, most education institutions in the United States and around the world had to transition into emergency remote teaching due to the lockdown. And at this time, most institutions had no choice. Faculty were under intense pressure. They had little time to prepare for their online instructions. Students were confused. And in response, many centres for teaching and learning rushed to create emergency online faculty training to support faculty.

Wai Ling
This moment in history was a moment of disruption and rapid adaptation. At George Mason University, we are no different. When the pandemic hit, during spring break, we went into emergency lockdown on March 17. And very quickly, the Centre for Teaching and Learning at George Mason created the online course design primer as a way to support Mason's faculty to transition online. The primary goal of this module was to help faculty learn best practices in online teaching, gain confidence in building an online course, and to become more independent as they navigate this very uncertain territory. While the pandemic shaped my research, the idea for this research question came really much later when I had the opportunity to reflect on how unique this moment was in higher education.

Wai Ling
We knew that many faculty members enrolled in the online course design primer. And then over the years, we transitioned from online to hybrid and now back to in-person. And for the past two years, I kept asking myself, how did participation in the online course design primer influence faculty's teaching practices over the long term? I found this to be a very unique question because for the first time, the Stern Centre for Teaching and Learning had to support and unprecedented number of faculty who sign up. And this has never happened before. And my question was really, with this big group of faculty signing up for this course, what is happening and how has it impacted their teaching practices in modern day?

Wai Ling
And also from my research, I understand that there is a significant gap that usually happens after training ends. And most professional development training are focused on short-term evaluations. They are focused on their initial reaction and satisfaction. They tend to be quantitative because these evaluations have to be quick. However, they tend to neglect the longer-term effects of a program. And there's also a lack of behavioral and organizational impact, as well as taking out the contextual factors, like, for example, post-training environment or support, that really goes to evaluate how effective a training is. However, long-term evaluations offer more potential to understand the longer-term influence.

Wai Ling
For example, it's always designed as multiple touchpoints assessments. It means that we check in with faculty not only after the training, but maybe a few months down the road or even a few years. It's designed to assess sustained impact on their teaching skills. identify long-term changes, and there's also the possibility to understand if there is an impact on organisation and institution culture. So how I go about to conduct my dissertation research is that I opted for a longitudinal qualitative case study that is exploratory because this is a very new phenomenon. It's intrinsic because it is driven by my own motivations to understand the case.

Wai Ling
And I'm using data from 2021 and 2024. The research site is entirely here at George Mason and I use thematic analysis to understand the data. As a novice researcher, I needed a framework to help me make sense of the data. And I opt to go with the Kirkpatrick evaluation model, which is a common program evaluation methodology that is used in both academic and non-academic. So very broadly, it looks at four levels of a training. The first level examines if the learners have a good favorability from the training. The second level asks if the training was successful in terms of transferring knowledge.

Wai Ling
The third level asks if the training made any changes in the behaviour. And finally, level four evaluates if the training made any difference on the bigger organisational impact. So going back to my research question, which is how did participation in the online course design primary influence faculty's teaching practices over the long term? I look at the first three levels and I will also explain that level four is not included in my study because this is the scope is too big, and I'm only focusing on faculty's teaching practices, and that's why I'm focusing only on the first three levels.

Wai Ling
So to tailor it closer to the model, I ask in level one, how did participants respond to the online course design primer? Level two, what knowledge or skills did they gain from the online course design primer? And level three, How have they applied their learning to their current teaching practices today? I have overall four data sources. So for context on the far left, the OCDP was launched in the summer of 2021. Sorry, summer of 2020. In 2021, the Stern Center actually conducted their very own post-program evaluation called the Extended Assessment Interview. And I'm very fortunate to be able to get my hands on this data to look at it.

Wai Ling
It's both close-ended with structured questions and open-ended qualitative interviews, and these are all self-reported. And then in 2024, I decide to conduct interviews with the same participants from 2021, also qualitative interviews. And for additional data sources, I reach out to the facilitators from the online course design primer, as well as writing my own researcher's memo to track the evolution of my thinking. In total, the number of participants that signed up for the 2024 interviews were 14 participants. And their motivations to participate in the online course design primer was they had no prior online teaching experience and they wanted to learn how to teach online.

Wai Ling
Now, there were three modules that were offered in 2020. However, I'm only selecting participants who did the summer and the fall of 2020 because these two modules shared the same amount of modules. There were seven modules. The earlier module had only five. So to standardize the same experience, only the fall and the summer were selected. And these participants come from all different rankings across the university, as well as they were teaching differently, they come from different disciplines. Very broadly, my research phase is divided into four. In phase one, I call this the familiarization with the initial data.

Wai Ling
Because the 2021 data did not come from me, I needed to really make sure that I understand what I was looking for. So what I did was I conducted the first cycle coding with all the participants. There were a number of 36 participants from both summer and fall. So this true inductive coding, which is a bottoms-up approach, I get to just get a sense of what participants were talking about and just get to know who they are. Now, when I move to phase two, this is when I start my data collection. I have to also say, at this point, I do not know who is going to respond to my call for interviews yet.

Wai Ling
But after I conducted my interview reach out, and then when participants start to pour in, and what I did was I returned to the same participants in 2021 just to have the ability to compare their experiences in 2021 and 2024. And in phase two, I also conducted interviews with the facilitators. So in phase three, this is where I begin my data analysis using the Kirkpatrick evaluation model. I needed to make sure that I am staying loyal to the model and using the framework to understand what I'm looking at. I also conducted member checking, where I send the transcripts from 2024 to my participants and have them make sure that I'm quoting them correctly and I'm understanding their experiences, as well as conducting the triangulation with the modules from the OCDP and as well as the facilitators.

Wai Ling
And finally, in phase four, I integrated these multiple data points and I wrote the longitudinal patterns that I could see from both years. So the findings, very broadly, these were the themes that emerged from Level 1. So when I asked participants how did they respond to the OCDP, they provided their satisfaction and their feedback. They also quoted three other elements that really pipe in to communicate why it was favorable to them. So the first one, participants in both years reported that the OCDP was highly well, was well organized and they appreciated the timing. They also clarified that it helped them with their online teaching practices as well as they value the templates and resources.

Wai Ling
However, one of the feedback that they provided was that they were frustrated with the workload and the tedious assignment. When it comes to facilitators, they highly value the timely feedback that was provided. They also mentioned that the support was personalized rather than giving a very generic blanket feedback. And this helped them to really create individual courses that help their students. And finally, they also mentioned that facilitators were well available beyond the OCDP, and they really appreciate this. They mentioned that the course content was highly relevant because it helped them, the OCDP provided a reliable model that they can emulate in their own teaching.

Wai Ling
12:52 - 13:35
And it also helped them as online learners to understand what students were going through. And that really helped them to become better online instructors. And finally, they mentioned that learning through community was a highly valued experience because first, it made them feel less alone. They enjoyed the engagement with their peers because it allowed them to ask questions and get feedback from each other. And more importantly, they were learning with others in the same discipline. Yeah. Anyway, I decided to stop here because I realized it's only five minutes left. Yeah, so I think even though there's more, let's just have a discussion.

Greg
Thank you. That was absolutely wonderful, Weiling. What a great presentation. We're going to go around the room and ask for any feedback to Weiling's presentation, if we could just keep it to the content of the presentation. And Alessandra, I'll keep it short, just a couple of sentences. Alessandra, do you have any feedback?

Alessandra
I'm going to leave the floor for everybody else here, because I got to talk to Weiling a little earlier.

Greg
14:01 - 14:04
Hilary, feedback, questions?

Hillary
Thank you, Greg, and thank you, Weiling. That was fantastic. What I liked was, I want to say the simplicity of what you're telling us. I understood what you were saying, and I appreciated that. You gave context to the value of this by looking at it in this way. You were bringing that up at the beginning. And I loved that, you know, how people don't look at these long-term effects and, you know, lifestyle changes that can happen from these things being present and the presentation. Yeah. Good info.

Wai Ling
Oh, thank you.

Devin
Thank you, Hilary. Devon, thoughts or feedback? Sure. I would just say that something we've talked a lot about in this group on various topics is the importance of story, that we all sort of learn things better and pay closer attention when it is in the form of a story. And this was, and I know that because when you had to stop, I was disappointed. I wanted to say, but what happens? So, yeah, great job, Eileen. Thank you, Devin. 

Greg
Michael, thoughts or feedback?

Michael
Yeah, I agree with what was said before about that it was a nice story and I wonder I wonder if it's relevant to bring something forward, give a little bit more of a taste of the conclusions, maybe at the beginning, I don't know, to grab something. I mean, I know it's kind of an academic context a bit more. So maybe the expectation is kind of like, okay, questions, methodology, data analysis, results, conclusions, but I feel it can still be valuable to kind of bring something ahead of time. A little, a short summary overall with some engagement of also the conclusions up front could be also engaging.

Wai Ling
15:55 - 15:57
Thank you. Thank you. I like that. Yeah.

Greg
Thank you, Michael. Bobby, for feedback. 

Bobby. B
The amount of time you spent discussing each slide versus the content on each slide was really good. You know, also none of the slides were had too much contextual information or complication such that my mind was now trying to understand the slide versus hearing you. There was a really nice balance between level of content, level of use of color and graphics relative to what you were saying. So I really loved that. As to how you shorten it that'll be interesting you know because i said it was just really really well done i didn't get to hear the very beginning but as michael said you know there might be that little you know i'm going to talk basically i'm going to be talking about this i'm going to touch on.

Bobby. B
16:48 - 17:35
the you know the findings along the way both you know including unexpected and and twists and you know maybe something really brief but as pointed out this is an academic presentation they're expecting you know, the academic presentation, and you're doing a great, and I'll touch on what Devin said, there's a beautiful level of inflection in your voice when you're talking about this that doesn't sensationalize or turn it into storytelling, but yet is enough that the audience is understanding your, the emotions in your words, where they matter. So end to end, I think, yeah, you've really got a great one going here.

Wai Ling
17:36 - 17:36
Thank you.

Greg
17:37 - 17:40
Thank you, Bobbie. Gretchen, did you have any thoughts or feedback?

Gretchen
17:40 - 17:44
Pretty late, so I caught just the end.

Greg
17:44 - 17:47
Thank you. Sharon, do you have any thoughts or feedback for Wai Ling?

Sharon
17:48 - 17:50
No, I think the presentation was great.

Greg
17:50 - 17:53
Thank you, Sharon. Shadows, any thoughts or feedback?

Shadows Pub
17:54 - 18:04
I came in a bit late. What I heard sounded good. I wasn't paying too much attention to the slides. For the most part, I either listen or I read. In this case, I was listening. So it sounded good.

Greg
18:04 - 18:28
Well, I appreciate your willingness to be vulnerable and open yourself up for feedback from everybody. It's very brave of you. I really enjoyed the content. And I'm going to have to try and hit you up for those slides because they were absolutely beautiful. I know that on the podcast, you can't see the slides, but they were some super-duper slides. The colors, the contrast, the way that they were laid out, that was great. Alessandra, closing thoughts?

Alessandra
18:29 - 19:27
Well, and I grabbed all of the slides that I know that there's the time is a ticket between now and the time that Wiley does this presentation for the committee and in just a few days time. So I grabbed the ones that if we have. the leisure of of time to look at something together. I grabbed some of those slides so that we can we just give them a little tweak. What I found as a designer who's also gone through the teaching and learning education things with for professors and educators is that the more that you can unify how the information looks from slide to slide to slide to slide the more quickly that It can be parsed out and more quickly the learner in this case, your learner is the committee, the more quickly the learner can encode that information.

Alessandra
19:28 - 19:54
And because what we're doing is we're. When we're learning, we're throwing things into a couple of buckets. One is, I know that already. And the other is, oh, what am I learning here? And so when you unify the graphic design across the slides, it makes it really, really easy and fast to throw the information that's coming in. Like, I know that, I know that, I know that. I'm going to look at that one. I know that, I know that, I know that. I'm going to grab that one.

Alessandra
19:55 - 20:28
if I can help with that, that'll be it. And it's always fun. What was really fun, Greg and everyone, is that Weiling went to, she presented on graphic design for the presentation of academic posters. And that was only two weeks ago, right? Wasn't it, Weiling? And so we went through with, we went through what she was going to present in the same way. So she and I already know how to work together really quickly on academic slides. So how do you feel right now, Wai Ling?

Wai Ling
20:28 - 20:48
Yeah, I feel like, so I think I was presenting about 15 minutes, was I, about that? I think it's going to be so tight, like even if I were to be given 25 minutes. So I think my challenge now is like more succinct. Like, yeah, I gotta think about it. Yeah, that's my concern.

Alessandra
20:51 - 21:28
Well, we appreciate you for this portion. This is the first section of what we are doing to celebrate your becoming, not just becoming more you, but literally your name changing to Wai-Ling Fong to Wai-Ling Fong comma P-H fricking D. So we're going to just pause the recording right there so that we can get on with creative work hour. But let's give our girl a hand. Yay! Oh, thank you so much. It's a lot of visual applause. A lot of visual applause.

Greg
21:29 - 21:36
Well, what happened again? You wasted some perfectly good time listening to the creative podcast. But watch out for the second installment of this presentation.