OEG Voices Episode 83: International Review of Research in Open and Distributed Learning (IRRODL) at 25 === Intro Music and Highlighted Quotes --- [00:00:06] Connie Blomgren: think it is a beacon of hope in a world where we do need more access to good quality research, articles that help people think and extend and grow and work towards authentic equity. [00:00:25] Adnan Qayyum: And so we see these waves that Connie was talking about. Aga and Terry mention that a lot of it is just what are people submitting. Those submissions do indicate a certain sort of practice of what's happening in the field and what is actually the conversation and the distributed learning challenges of the time. [00:00:44] Aga Palalas: So what I'm saying is it's alive. It for sure is, and it's surely is very popular amongst researchers and potential article authors. [00:00:58] Terry Anderson: Two features of IRRODL that have been there since day one, and I think are critical to its success. The first is Athabasca University's support of it. I've talked to lots of other journal editors and they would love to have a full-time managing editor, and IRRODL has always had at least a part-time and almost the whole time, a full-time person. [00:01:21] Alan Levine: You won't believe listeners, if you're listening to this, what it took to get this recording and podcast ready. Special Episode of OE Global Voices --- [00:01:28] Alan Levine: But we're very glad to have everybody here for a very special episode of OE Global Voices. [00:01:34] This is the podcast that we do at Open Education Global and we just want to bring conversation style, the people and the projects in this world of open education and talk about 'em, not like interview, but we just want to act like we're sitting around a table. [00:01:49] And I'm really happy to be here today for this conversation. I really want to acknowledge our colleague and OE Global board member Connie Blomgren for suggesting we talk to some of the folks-- at now we did the name check, the International Review of Research and Open Distributed Learning. I checked my notes here too, because-- how do you say it? Is it "eh- Roddle" or "Eye Roddle"? What's the official name? Somebody. [00:02:18] Terry Anderson: I always say "Ih-Roddle" [00:02:20] Alan Levine: Okay. [00:02:21] Aga Palalas: It depends who you ask. [00:02:25] Alan Levine: That's good. That means IRRODL's an interesting entity. [00:02:29] We're gathered here because it's the 25th anniversary, and that alone is-- you have to stop and say like-- 25 years of open access publishing, 25 years that you can go to those-- I just went to the first, the first volume and they're still there on the web. And, I don't know about you, but I get sad out at how much of the web has been disappearing on us. [00:02:50] We're gonna talk a lot about that and also about what, the future looks like for IRRODL, but the first important thing for me to do as the host is to shut up. And I want to give a chance for our colleagues here gathered, to say hello. And what, I ask is if you let us know where you are, and you'll probably give a map location, but then since we're on a podcast, I always ask if you can describe your physical location-- like what, do you see in your room. [00:03:19] And so I will start first with, Connie, because she's responsible for this and also because we, get to work together. And then I'll ask you in turn to, to pass the intros around the room. [00:03:32] Welcome Connie. Great to see you again. [00:03:34] Connie Blomgren: Oh, thanks so much. It's always great to be here with you, Alan, and always ready to support open education in its various formats, Open Education Global, IRRODL, and all the good work that's been going on for so many years. Land Acknowledgements and Personal Introductions --- [00:03:52] Connie Blomgren: I'm gonna read out my land acknowledgement. [00:03:54] So I acknowledge and live in the present day Treaty seven region of Southern Alberta, the lands of the Niitsítpiis-stahkoii-- my apologies there, I'm just learning Blackfoot-- Blackfoot Nation. So my home is in the traditional territories of the Siksika, the Piikani, the Kainai, the Tsuut’ina, and the Îyârhe Nakoda nations. And this area is also part of the Métis homeland. [00:04:23] I take personal responsibility to understand our shared history and my personal role in conciliation and reconciliation. Nitsíniiyi'taki I am grateful for our shared living environment [00:04:38] And this morning it's a beautiful May day. We've had a kind of a cool spring and last night it rained, so it's always a good thing because it's-- I think it's dry. I know it's dry and we didn't get enough snow this year as we should, so it's always good to have a spring and rain in the spring. [00:05:01] So yeah, I'm here in the beautiful, sunny, southwest corner of Alberta. So I'll pass the microphone over to Terry. [00:05:10] Terry Anderson: Okay. My name is, Terry Anderson, and I'm here in Edmonton, Alberta on a beautiful spring morning. I'm currently retired. I'm the professor emeritus at Athabasca University and professor -- err editor emeritus, from IRRODL. I'm really glad to get a chance to meet new friends or remeet of people. Again, I'm a little bit much more isolated these days from the wild world of online publishing, so it's really nice to connect, base and see you again, Alan, after all these years. That's it. next to Aga. [00:05:52] Aga Palalas: Oh, I was nominated. Thank you, Terry. So in my travels between Alberta and Ontario, today I am joining you from Southern Ontario, the town of Oakville that is located on the treaty lands and territory of the Mississaugas of the Credit. It's the sacred lands of treaties 14 and 22. And I do acknowledge and really, thank the Mississaugas of the Credit First Nations and the treaty holders, for being stewards of this beautiful, traditional territory and the beautiful lands, surrounded by the Great Lakes. [00:06:47] And I really enjoy looking at the beautiful nature from this basement office. My window looks out onto beautiful really established old trees that carry a lot of history and ground me every day. As I work, as, I work-- as a "prisoner" of this digital space, I get to enjoy the beauty of nature. [00:07:20] So thank you very much. I am a newcomer to IRRODL, in a very, what I consider, essential role as an editor in chief acting and really enjoying that role. [00:07:37] I've been with IRRODL all its years as a reader. [00:07:40] Thank you and over to Adnan. [00:07:44] Adnan Qayyum: I'm also a novel, to IRRODL. My name's Annan Qayyum. I've been at Athabasca for about five years now, and IRRODL's always been a part of my life. I never talked to Terry about this, But, since its early days, I've always been delighted to be having an engagement at IRRODL. [00:08:02] I remember when Terry had an mp3 format, way back when, decades ago, and I was working in the US and I was actually working at Penn State. And-- I would, I was still coming back and forth to Canada at the time because we didn't have paperwork for my family. And I would listen to IRRODL on mp3 as I was driving. So when you changed formats, Terry, you didn't have that anymore, I was like, oh, it's exactly one of my, early podcast listens way back when. [00:08:28] So it feel to me like full circle of podcasting about IRRODL instead of listening to IRRODL in a podcast format. [00:08:36] Alan Levine: Oh, that's fantastic. Thank you everybody. [00:08:37] And, like the connections and whether you're, new and, I'm probably the newest in, the room to IRRODL. I've been aware of it, but and so that's-- there's so many places to go. We want to talk about the history. [00:08:50] And I just want to also add, I am almost adjacent. I'm actually, I moved to, Saskatchewan in 2018. I too am very appreciative of living on treaty 4 territory. And the land is so special to me. Today is-- we have an outbreak of weather, we have rain. Connie and I often talk about, 'cause we live in similar terrains. But there's something really about having that like personal attachment and meaning that the change in weather like affects us and we have such glorious sunsets. I too am very appreciative of just getting to be here, right now. Reflections on IRRODL's Role and Future --- [00:09:28] Alan Levine: And to talk about IRRODL, like I'm a little bit torn, logically we'd start at the beginning and we want to get to that to the roots, but maybe what, does IRRODL look like here in 2025? [00:09:41] What's the state of IRRODL as an open access research journal? [00:09:45] Aga Palalas: Okay, I'm gonna give you that fresh, unbiased look. [00:09:50] It looks good. It definitely is still that very same prestigious open access journal that I know I've been always relying on as a researcher-- as definitely as a professor and a person who guides doctoral and master's students in distance open and digital education through their learning and research. [00:10:22] And as a person who advocates accessibility, diversity, sharing voices around the globe. And, so IRRODL is all of that right now to me. What it also is, through my new perspective, from my new perspective through the lens of a person that's involved in the editorial re-- sorry, reviewing and then editorial process. [00:10:57] It's also a lot of work. [00:11:00] So it's a full living, a really flourishing entity that has taken a lot of space in my life. And I hope that, this is what it also means to the many people who have been-- have been following it and benefiting from submitting their articles as well as reading, sharing and the dialogue around it. [00:11:29] So what I'm saying is it's alive. It for sure is, and it's surely is very popular amongst researchers and potential article authors. [00:11:41] Alan Levine: Sure. Add then I see a finger raised. Let's hear from you too. [00:11:47] Adnan Qayyum: I've actually had the fortune of working in other journals as well in the field, in the US, in Australia. In a time when I find myself really worried about the nature of knowledge and the value of knowledge, in a time when I worry about the value of ethics, IRRODL stands out for me the bit of hope. Because I've been with other journals and early on Terry and others who were involved, initiated this idea of this open access journal. [00:12:20] And I've another involved with other journals which were not open access, which were at times proprietary, times pay to play. And IRRODL has the won the day in terms of the field. And to me that's remarkable. It's remarkable that it has because it is still the most prestigious journal in the field. It's the number one education journal in Canada. And I like to flex that in conversation regularly at the way of flexing it for Athabasca. [00:12:42] And so to me it's a really hope that the good guys can win. Yeah, really, I, mean that, you know, and, so I'm genuinely really humbled to be a part of this, being part of Athabasca, but being a part of the journal and being somebody involved. [00:12:56] And of course, absolutely what Aga saying, right, it's very alive. The internal machinations of the sheer volume of reviews we have to do and the number of people want to get published in here. But really like that to me is I feel really-- I've come to it late, but I'm so happy to be a part of this because it's oh, this is great. The values I want to believe in, it can actually succeed. [00:13:17] Connie Blomgren: Yeah, I would say, I've been an associate editor at IRRODL since 2018, I think. I've seen some changes over time and it was interesting to see IRRODL handle the pivot to online when there are a lot of people unfamiliar with the theory and research in distance education that's was going on for many years before the pandemic of, 2020. [00:13:49] And then all of a sudden a lot of research papers coming in around that and talking about the pivot. And our reviewers were always very gracious, I thought, in saying, okay, let's help you out here. There's a big body of research that's been going on in this area of online education. [00:14:09] But the history of online education, of course, goes back to the history of distance education, which is actually quite an old history. And it's really about accessibility to learning in different ways at different points and flexibility. Of course in this day and age when we see such unusual things happening in our world, and some of it related to education, to funding, to research, all of those important questions of what IRRODL is about. [00:14:40] I think it is a beacon of hope in a world where we do need more access to good quality research, articles that help people think and extend and grow and work towards authentic equity. [00:14:58] So it's a big goal. I think it's always been very ambitious. . I think IRRODL-- I think it's interesting this role of the imagination to think of 25 years ago when the internet was still relatively young for most people. There were all sorts of possibilities. And I think Terry, that the imagination of we want to make something open and accessible to people wherever they are. [00:15:32] To me, that's quite a vision and it's exciting to just be part of that vision. So I think it was a true act of imagination and reality coming together when IRRODL first began. And, we continue obviously for all sorts of good reasons, but I would say it's alive and well because there's lots of people are very committed IRRODL in different ways. [00:15:56] So that's what I think right now. Alan, big question. I'll pass the mic to whoever wants to talk. [00:16:04] Terry Anderson: Let me say two things that, or two features of IRRODL that have been there since day one, and I think are critical to its success. And the first is Athabasca University's support of it. I've talked to lots of other journal editors and they would love to have a full-time managing editor, and IRRODL has always had at least a part-time and, almost the whole time, a full-time person. [00:16:28] So that's a contribution that comes right off the top of Athabasca University's budget. And they've never really complained or told us to cut back or done much other than continue to pay a good salary. So I'm really grateful for that. [00:16:45] And the second thing is that we were very early pioneers with Open Journal Systems which now hosts probably 90% of the open access journals around the world. And it was developed at both Simon Fraser University and University of British Columbia. But it's an incredibly powerful tool, and it's-- and I was looking at going through it for the 25th anniversary issue that I edited and noticed how much more complex and how much more interesting the OJS system is and the kind of data that it can get on past issues and things like that. [00:17:23] And not only the managing, the whole intake, the reviews, but also the publication, the look and feel and all that other stuff. So, it's been a real asset to the whole world that would actually be worth an interview to those folks too-- on open journal systems and open journal books. I think they have two products. [00:17:46] Alan Levine: And probably there's so much to it that we don't even see from reading the journal. Already, each one of you provided a quote that I want to use to summarize this. I was noting in just reading the most current issue, like the emphasis on international how well IRRODL and throughout the series, the history of focusing on regions, but just the represent-- representation. It's just not the same group of researchers that you always hear from. [00:18:15] So I would gather it's a great opportunity for new upcoming researchers, to make their work known. Is that what you're seeing in terms of the submissions that come in, that you're really tapping into, this global community? [00:18:32] I see Adnan with a eager hand up. [00:18:37] Adnan Qayyum: On that note, I want to clarify this anecdote that I share with people. And I want to see-- ask Terry if it's true or not. [00:18:44] In my mind, I remember this story about some young academic publishing in IRRODL many years ago, like maybe 10, 15 years ago, from China. And they approached you and they said, Terry, I gotta-- because I got published in IRRODL, I would actually given an apartment [00:19:00] Terry Anderson: I think that's still the case. [00:19:02] Adnan Qayyum: And the point being like this was the kind of impact it had on young researchers in different parts of the world. If you could get published in IRRODL, it made a big improvement not just in your career, but in your life-- in a way that I didn't realize. And so I would share that story anecdotally, Terry, with some of my doctoral students at various universities, they're like, "No, that can't be right." [00:19:22] But if you could confirm that would be great. [00:19:24] Terry Anderson: I know that's exactly right. It's-- and it's part of the monopoly of the SSCI journals, those ones that are recognized and why we fought so long and so hard to get that recognition. [00:19:34] But once you get that-- like for a lot of people in China, I spent quite a bit of a number of times, months in China, and it seemed like for them, it was throwing an article away if you didn't get it in an SSCI and why we allow Thomson Reuters to control what's quality, by that index. I don't know. [00:19:55] But nonetheless, we played the game,. We won and we got admitted to this elite crowd, and it has made a big difference in the quality and the quantity of submissions that we get. Global Reach and Inclusivity of IRRODL --- [00:20:11] Aga Palalas: On that note, because you did ask if we still getting submissions from all over the world. And the short answer is absolutely yes. And I'm very proud of the fact that the Global South is represented as well as it is in our submissions. I think there is a number, I was just looking through the article-- what was the number? We have been-- I think 31 countries is where the published articles come from. [00:20:43] And, I would like to speak to one issue that nobody else in the room might not even consider the importance of. But being an ESL speaker, being a person who speaks English as a second language and who lives with that my own, I guess self-imposed restriction of, "Oh my gosh, can I express myself the same way everybody else can,-- Can I submit to prestigious journals like IRRODL?" [00:21:18] Yes, you can. The conversation about how to support English, non-English speakers at IRRODL has been there forever. I've been part of the conversation even when I would think 10, 15 years ago when I was not involved so closely with IRRODL, but was considering publishing,-- not publishing-- but submitting to IRRODL. [00:21:45] And this is really beautiful to be seeing how the journal has been open to people who communicate in other languages than English and then go through the process of expressing themself in English. And how now, we have transitioned to supporting all kinds of AI applications, in a proper way, in an ethical way to, support those that need supports to express themselves and report their unbelievable research in English. [00:22:26] So I think that's a very-- maybe not everybody appreciates, but every time I read submissions from people, non-English speakers, I think of that and I put my hands together to applaud IRRODL. [00:22:44] Alan Levine: And we all do. I see lots of head nodding. [00:22:46] So I, again, like I-- I have 15 threads. I do-- I think it's worth it to establish the origin, and not to spend too much time focusing on that, but it's pretty remarkable. What was the impetus in 2000 to start this journal? Founding and Evolution of IRRODL --- [00:23:04] Alan Levine: And maybe clarify what was the form because I know it's gone through different-- like we talked about the different formats of audio and online, but why did it start? How did it start and what, did the first IRRODL look like when it was published? [00:23:20] Terry Anderson: Sure, maybe I can address that. [00:23:24] Some people think that I was the founding editor and I was not. Peter Cookson was the Vice President Research at the time, and he left Athabasca about a year after IRRODL was founded and then I became the editor after that. [00:23:39] Alan Levine: As an addition after the recording of this episode, we're inserting from the very first edition of IRRODL, the mp3 version of the editorial by Peter Cookson from the June, 2000 inaugural issue. [00:23:53] It appears this was computer generated, not a recording, but that's how IRRODL was early to the game. [00:24:00] IRRODL: I extend a warm welcome to you as a reader of this inaugural issue of the International Review of Research and Open and Distance Learning IRRODL. In this my first editorial, I explain the meaning of the journal's name, purpose, content, and structure. I also explain the theme of the articles in this initial issue and express appreciation for the many people who support has been instrumental in converting IRRODL from an idea to reality. [00:24:27] Terry Anderson: But I guess there were two things that-- which gave IRRODL real first early adopter or a first player advantage. And the one was of course, online publications because, previous to that there was a sense amongst professors that, again, you're throwing your article away if you don't get it in paper, 'cause then nobody will be able to find it. It's ironic now we can't find the ones that are in paper. [00:24:55] But, anyways, and it was decided, I think maybe a tiny bit because of the economics of not having to charge people and then not having to-- to print and the mail and all those other sorts of things. But it was also a recognition, Athabasca is and was an open university, was the fact that we were gonna try something different. [00:25:19] So that was certainly a big part of it. And, what? Oh, the other thing is that we are really at the beginning cusp of the change in distance education. Distance education has a long history, and Athabasca was like 20 years old and in the correspondence mode. And all the research on distance ed was this asynchronous, print-based, no real interaction except maybe at a summer camp or something like that. [00:25:49] And so all of a sudden we were at this place where, whoa, you could actually talk to another student in real time and you could have pure relationship, collaborative learning, even if it was just on audio. You were-- it was a whole new world. And, there was an explosion of interest with early adopters from the traditional universities. [00:26:11] So the audience was larger than the usual distance ed crowd, the open universities, who were dipping their fingers in. [00:26:19] For a lot of people thought, of course, that it was a sub quality of education. And so many of the traditional universities were a long time coming into it. In fact, it took Covid to bring some of the last laggards over the line. [00:26:33] But it was a time of real excitement in online learning because we had these new tools. We had these new technologies, and we are getting away from entirely asynchronous kind of learning, experiences. [00:26:50] So the first issue, Peter got a bunch of grand experts to add their face value to it, to write articles. And I, don't recall whether it was very it-- Peter was a perfectionist and he edited it. But there wasn't much peer review. We always thought it would be a peer review, but the very first issue, we were probably a little shy on that department. But if you read the original authors, they were all well-known scholars in the area. [00:27:24] So they endorsed the very first issue by their presence and by their contribution. [00:27:31] Alan Levine: I wanted to see the first issue and I was reading, there's an article on digital learning environments and new opportunities and it's it like wow. It's like they were really thinking about this and it wasn't just about like we're talking about yeah, distance learning or the tools or the platforms and that, that runs through the history. [00:27:50] And interesting to note that point where it was decided to change it from "distance learning" to "distributed learning" is fascinating. Was there a lot of contention over that decision? [00:28:04] Terry Anderson: I don't think so. It was greased the wheels by Rory McGreal getting a grant for distributed kinds of learning from his UNESCO chair. He said, they wanted to see what really would actually get for the money. And they got a name change in IRRODL. And that was overdue anyways, because distance education had-- has a checkered history in terms of its name. People have not liked it for a long time, 'cause really, what is the distance? Is it time, distance? Is it-- what does it mean? [00:28:40] And, when especially in Athabasca and the vast majority of the learners are in major cities, they're not out on the trap line and in real distant places. They're actually not far from the university in some cases. So. I think that explained that little change, and it was nice. We could keep the same initials, broaden the environment. [00:29:03] Alan Levine: And I just also want to maybe if you can talk about the format. 'cause I noticed like the structure in the beginning of the main section of articles, the part of Notes which are just relevant information and this idea of Web Conferencing that tried and maybe it didn't work out. But I was in intrigued to think of that you wanted to introduce a way of having dialogue with authors early on. [00:29:29] Terry Anderson: Yeah. Yeah, we did. And one thing you'll notice when you talk about the different formats over the years, IRRODL was always-- it was easy to change things and to try something, We didn't have to have a strong editorial board, really. It was basically the few people who were doing the actual editing on the fly, just changing. [00:29:53] And so we added different audio and different formats. We tried to have more interactive where people would review them after publication and the authors could get back and defend their work or this or that. [00:30:08] And that was a big dud. People just thought, I think they thought, I got this thing out the door. It's published. That's it. I'm not gonna spend another 10 hours trying to justify it to the whole world or something like that. [00:30:23] And also the reviewers, people maybe they were fatigued from reviewing it in the first place, but it could have opened up to new audiences, but it never really did. [00:30:34] And that's fine. It was worth an experiment. [00:30:38] And same with publishing that we used to-- we talked a lot, it is this batch waiting till you have a whole issue putting them all together. It doesn't dilute the value when you have to read 20 articles. What if you could read one a week kind of thing. [00:30:53] So we tried that too. And, it again maybe it's just the tradition of what an a- an art-- what a journal issue is, and made it so that we, kept with that more traditional way. [00:31:07] And then that's why we too experimented with special issues. And that was always fun because then you could draw in a new editor to take control. [00:31:16] There's always been a hassle though, when you have a special issue is who is in charge? Is this guest editor or is the editor in chief? Or what if they accept some articles and you wouldn't, and vice versa. And, so it creates some tensions from the normal, but, I think it was well worth it. [00:31:37] Alan Levine: How do you come up with the themes or the topics for each one? And, are you, can you say what is coming in the future? How do you pick what you want to frame a new issue around? [00:31:49] Aga Palalas: How we pick, we pick the best! [00:31:52] Alan Levine: The current one is on the hot topic of artificial intelligence, but when you say it's time to do, something to focus on this region of the world-- do you try to set up something that people will target at, or do you read the field or do you just say it's time to address this topic? [00:32:12] Aga Palalas: So you're speaking about special issues? [00:32:14] Alan Levine: Yeah. [00:32:14] Aga Palalas: Oh, okay. On special issues, again, I'll let Terry and Connie speak about how that's been done in the past. I can speak to what it looks like today. [00:32:29] Knowing that or like emphasizing the fact that it's through the experimentation and all the pilots that Terry has just spoken to, that we have this very strong system that really we are following and it needs to be always refined. [00:32:51] Because for example, today of all the issues related issues and supports related to GenAI, but we do go with the same system, in terms of what's just happened lately with proposed special issues. We really looked at how well aligned with the special issue that's proposed be with our mission with what is expected of a IRRODL. [00:33:20] Is it really truly about teaching and learning in this open education environment and follows all its-- I mean it follows the vision, of openness, but really always looking for quality. And for the authors of the special issues, to be experts in the area of teaching and learning in the open distributed learning environment. [00:33:50] But over to my esteemed colleagues to how it's been done over the years and how we landed where we are today with all these great special issues. Understanding Special Issues in Academic Journals --- [00:34:01] Terry Anderson: I think the majority of issues are not special issues. And so they're driven by what's submitted and then the editorial editor either writes a general thing about some issue that they're interested in, the editor in the editorial and then does a paragraph or whatever on all the issues. [00:34:21] That's probably 80% of the articles. When a special issue, at least in my time, it would be driven mostly by some expert or another emailing and saying, "I'd love to be an editor." And then if it was a hot idea, we thought it was a good idea, and we had thought that person had the competence and the connections to actually get quality authors in, then we would, say yes. [00:34:50] But I think we learned quickly that we had to keep control of the editorial process. And that if this special issue was not working out and they we're not getting good articles or other whatever, we really had to-- we didn't want to feel like we had to publish it because somebody has had this idea. [00:35:12] But in any case, it didn't happen often. But, they offer opportunity and challenge for the editorial team, a special issue. [00:35:21] Connie Blomgren: Yeah, I would say special issues are "special." And so they come generally an email or sometimes I think could occur from maybe attending a conference and people approaching Rory or Terry or other editors and discussing the potential idea of that. Waves of Research Trends in IRRODL --- [00:35:47] Connie Blomgren: But the bread and butter of IRRODL is the regular issue. What I see with those regular issues, which is interesting, is that you'll see waves of what's happening out in the field, because you'll get quite a few-- like when I first came to IRRODL I am, I'm not a MOOC researcher, but there were a lot of, submissions around MOOCs, lots coming in, and then that seems to have died out a little bit. [00:36:18] And then I saw a big Community of Inquiry wave. And then recently , it's interesting, but we've been seeing a wave of online music education submissions, which-- or subject areas may, it might not be dance was the other one too, music and dance online being taught and the research around it. [00:36:45] And so it's like you get bigger waves, smaller waves, and so that's not a true special issue, but you'll see, sometimes clusters of topics as certain things seem to become the buzz people are all interested in it. And then they shift away to some other things. And our journal is a reflection of what's happening in that broader environment in open and digital spaces. [00:37:15] I think it's interesting to just watch those waves and see what's coming. And of course, AI right? It's the one that's probably gonna be around for a while for all sorts of different reasons. [00:37:29] So yeah, it's very interesting to just see the topics emerge organically within the regular journal itself. And then of course, special issues being that focused attention. [00:37:40] Adnan Qayyum: Connie said-- Connie already said it. [00:37:42] I was just gonna say that I've always been curious about the relationship between what's happening out in the field and what the journal is publishing. And I think there is that relationship. I think about IRRODL and other journals, at one point, everything around instructional design was really big. [00:37:58] Lots of stuff on ID and them Quality Assurance became a very big topic and Student Support and then interaction, whether it was COI or other models, and then MOOCs and Open Education. And so we see these waves that Connie was talking about, and I think it is interesting that, Aga and Terry mention that a lot of it is just what are people submitting. But that's, those submissions do indicate a certain sort of practice of what's happening in the field and what is actually the conversation and the distributed learning challenges of the time. [00:38:29] So Covid certainly had its moment as well. And, yeah, I did not expect online music education to be a phenomena, but that's really something we've received a lot of recently. Yeah, it's notable that separate from the special issue, there is this sort of thematic kind of component to what happens with the journal. [00:38:49] Aga Palalas: And it's interesting how just observing that, witnessing that, and not only witnessing being involved. With the review process, you start-- we have conversations around these trends, right? So we get to learn and we are inspired to look into whatever it is that the submissions reflect, as the trends current in different parts of the world, right? [00:39:16] So we noticed that this is very much-- for example, there were moments when I was thinking, are we still open and interested in publishing on MOOCs And I had to really reflect on that only to answer to this question being, yes. Because it is so relevant for so many people in other parts of the world. And obviously there has been-- there has been transition into new ways of doing this. [00:39:50] It's very interesting how we as the insiders of IRRODL learn from all the submissions that come in. The Intensive Review Process --- [00:39:59] Alan Levine: Can you give the audience a sense about like how much, how, many papers do you get? What's the submission volume that comes in? It was on a chart, is it going up? [00:40:10] Aga Palalas: It is going up for sure. [00:40:14] So we have various levels of screening. Not, obviously not all the submissions land on reviewers' tables or screens. But I think that I always look at the average of, I get five, six emails of something just come in daily. [00:40:39] And at this point I think we are looking at, after a long process of those different levels of screening and then reviewing. If the submission , the manuscript makes it to that point we end up at this point, publishing 6 to 7%. And I think there are moments that we are at around 5% of those that were submitted. [00:41:04] Yeah. [00:41:05] Alan Levine: That's remarkable. It's all coming to you. That's ideal in some ways. [00:41:09] And, the problem that you have to figure out-- to me, like reading like a hallmark of this is the intensive review process. Like I saw in the FAQ is one of the questions is "Why is it taking so long to review?" [00:41:25] And I would imagine it's because it's done so intensively. Can you tell us about how the review process works? [00:41:32] Connie Blomgren: I think that's a great-- great point. Because I wanted to make the emphasis that IRRODL is possible because of author submissions and also reviewers. And as we just noted, there's a lot of submissions coming in. And some of that pre-screening is-- we get submissions because it is a high impact journal. [00:41:57] We get submissions from quite a few authors who, maybe tangentially are talking about something related to digital education, but it's not really within the scope. Not everything that is submitted actually fits in the scope. So there's different screening that happens. [00:42:18] But that said, once it passes through those different pre-screens, it's assigned by topic area to one of the associate editors. [00:42:28] And there we have another associate editor, Dietmar Kennepohl, who's not with us on the call today, and then our managing editor-- and big shout out to all the incredible work that Serena Henderson has done over the years because she is the glue that holds IRRODL together. And she's more than the glue. She's just really so knowledgeable and effective in being that managing editor. [00:42:55] So it's assigned to one of the associate editors to, then again double check, make sure that it aligns with the scope. And then we look at our reviewing process and contact and send it out. And through the OJS system, there's a lot of templates that are generated quite nicely. They're adapted for the IRRODL context, but the system certainly makes that effective. [00:43:24] And so it's sent out to reviewers. And, it depends on the time of year. During the pandemic, it was so hard to get reviewers to respond. Many of 'em were just so flat out busy being instructional designers or involved in online applications in their universities and colleges, that reviewing really was hard during the pandemic. [00:43:50] I was sensitive to trying to send it out in a good way because we do rely on that community of reviewers. And then they provide their feedback and through those decision making processes that most journals are using, there's a decision of to accept with no revisions, which is like rare. I don't know if I've ever seen that. Because academics always love to just suggest at least one revision. And so revisions and a variety of revisions that can be, or a decline or maybe a suggestion to submit to a different journal because again, it doesn't quite align with what IRRODL is about. [00:44:37] So it's a very intensive process. And it can take some time because we're relying on the unpaid work of scholars throughout the world. And we try to make sure that we're not requesting too many reviews from any one person in a year so that it is distributed in that way. So I just want to say thank you again to the reviewers. [00:45:02] It's a lot of work. they're very conscientious for the most part. We're always looking for good reviewers, so people, do sign up and yeah, there's a variety of people out there supporting the journal from that perspective. [00:45:16] So a big shout out to the community of reviewers as well. Challenges of International Reviewers --- [00:45:20] Adnan Qayyum: I'll just add that one of the challenges and opportunities in this journal is the fact that it is international. So getting reviewers, one of the challenges is the fact that we'll have people who are submitting from one part of the world, but the specialists we have on our database are from a different part of the world on that given subject area. [00:45:37] So I think that's also one of the features we need to keep in mind in terms of the process is to talk about online music education, right? It might have a submission from China, but the specialists we find are actually from, Brazil and Europe. And that's fine, but some of them will actually say, okay, I'll, address it. Or they might just hesitate, okay, I don't really know the context of what that meant within that country. [00:45:58] I don't know how it was earlier on when you folks did it before, but I do notice that because it is such a high profile and international journal, the review process has to include that the consideration. Innovations in the Review Process --- [00:46:11] Terry Anderson: If I could say one thing about the review process as well is that one of the innovations that we tried at IRRODL was to allow doctoral students to review, and it is supposed to be peer reviewed. [00:46:26] And I guess some people in some disciplines might think that a doctoral student is not appropriate. But through experimentation and actually reading the reviews, generally the student reviewers are much more thorough-- especially the reviewers who just check, check, and nothing real value do they add to the process. So we always get a good review from a student, and then they get a good learning experience as do all the reviewers. [00:46:54] Because OJS first allowed this to happen, we've been returning the reviews to the reviewers so they can see what the other reviewers said. Then the things they missed or like sometimes you'll think, "Oh, this is a terrific article", and then you'll read somebody else say, "oh, this article is crap. " [00:47:13] And you'll, have to say, oh man, is this me or is that him? And, and so there's a real learning experience for everyone to be a reviewer. And, yeah it's a bit thankless though because you can spend five minutes at it or you can spend five hours at it. And knowing academics are being pressed for increasing teaching loads as well as research expectations, it's always a struggle to find good reviewers, but, they're much appreciated. [00:47:47] Aga Palalas: On that note, I know directly from these doctoral students, how much they appreciate that opportunity. And I fully agree with that observation that doctoral students, especially those towards the end of their doctoral research, make really good reviewers. And we editors read all the reviews anyway, and we are in a position to make that final decision based on the reviews, and even evaluate the reviews themselves, which we do. [00:48:18] We evaluate the reviews and the reviewers, for our own information going forward as well. Mainly so this is important. And another note, our team also includes a statistician and a few copy editors. And these people are crucial to this process and they've been amazing. Again, a big shout out to them. [00:48:43] And again, Serena, without you, we would not be able to succeed and do what we are doing so well-- don't we, aren't we! And, sorry Serena cannot be here today, but I'm sure she is going to listen to this podcast. So thank you very much. [00:49:04] Alan Levine: That's good. I'm sure she'll love to hear that. [00:49:08] How hard it is it to make that final cut, like to get the final number of ones? Does it sometimes tear you apart or do you have- do you go back and forth and what does it take to get to the final list of published articles? [00:49:24] Terry Anderson: Yeah. I think one of the problems of the hard ones was when you get an article that's well written and you just think "there's just nothing new here." It's a good article and it would be a good article maybe in another journal, but this is IRRODL and so it's hard for people to get that kind of feedback too, is that it's, good work, but it's just not good enough or not exciting enough. [00:49:53] And that puts the personal judgment of the editor on the line too. And that happens if the reviewers are split or something like that. So, that to me was always a challenge. But, you had tended-- you have to be ruthless, especially as the submission rates started to increase. [00:50:11] It wasn't a question , in the early days, we used to publish whenever we had enough for an issue. And we were doing something like seven, eight issues a year and just burning out our managing editor like mad. And, so we finally decided this is ridiculous, and cut back. And so that meant the acceptance rates went down, and that's one of the hard things about a journal that gets a lot of submissions. [00:50:39] Adnan Qayyum: I'll just add that one of the sort of categories I've developed in my own mind when I see journal articles is-- you have a solid study, it's not a study which works here, it might work elsewhere. And it comes to the point that Terry mentioned is yeah, a lot of these studies have been done with technology A and context B for learners C kind of thing. [00:50:58] Okay, that's fine. If we've done that already and we've already had that, sort of conversation, maybe you go to another journal, which is continuing to have that conversation. But we're really looking, because we can be picky, we're very fortunate about that. We're looking for what's the important issues currently. And it comes back to that point around trends and waves, right? [00:51:17] Aga Palalas: We also look at, are the findings of this article really, in any way, informing learning outside of this particular case study or whatever the study is all about? Very often greatly written, very interesting, but would our readership really learn anything new as Terry said, or could they learn anything that would be applicable in their own unique learning environment? [00:51:49] And if the answer is no, then unfortunately, I personally sometimes shed a couple of tears and decline. [00:51:58] Alan Levine: Because you're the one who has to tell them. The Importance of Student Impact Evidence --- [00:52:00] Alan Levine: I was curious, I came across that the requirement for evidence of student impact, how did that evolve and what's been the effect of having that as part of the submission requirements? [00:52:14] Terry Anderson: It was after my time. I don't recall. It must, Rory must be at fault. [00:52:19] Alan Levine: That's fair. But it's remarkable to see in there 'cause I think, I know was getting there yeah, there's lots of things you can correlate and you can suggest or infer, but to say you actually want to see student impact, to me it just came across as this is something that it's more about publishing glossy and great sounding research, it's, supposed to be meaningful research. [00:52:44] Terry Anderson: I think if Rory was here, he's adamant that there has to be empirical results showing something. And he does not like the "everyone was happy at the end of the course", which is probably a quarter to a third of the research is just perception data and Rory loves to throw those ones out. [00:53:07] And I think we all do too to some degree. some of us are more-- more adamant about that. It puts a sneer to qualitative research, which just often does look at attitudes and, finds some significant, but I know Rory thinks that is education, this is learning. And if you're not evidence of learning, doesn't matter how happy you are, kind of thing. [00:53:34] Aga Palalas: And it gives that-- in a way it gives a guideline to our authors as well. However, it is an ongoing conversation amongst the team. I should mention that I see Adnan wanted to add something. [00:53:49] Adnan Qayyum: I think the reason that the impact feature is important though, is because of the field we're in. It's easy to get caught up with the newest, shiny, new object, right? Okay. This, new technology's amazing. Everyone's really excited about it. And to just get caught up with the conversation on the tech as opposed to what it means for the classroom, what it means for the institution's sense of the meaningfulness or the impact. That's why it's important, because in the world generally of education, certainly ed tech and partly distributed learning, it's easy to get caught up with the technology is the priority when and, absolutely there's some remarkable things that can do, but to keep the focus on why we're here as educators. [00:54:31] Alan Levine: Thank you. I have a billion questions. We had delay because of some technical snafus. I kind of want to ask 'cause Adnan brought up the surprise at seeing music education. Highlighting Unique Research Contributions --- [00:54:44] Alan Levine: Is there like a paper that came in that just, that stands out for you as something that just kinda knocked your socks off that said, "I didn't think there'd be research on this" or this is really wow, like, I want to send this to someone else. or, just I didn't think we'd be publishing on this much on music education. [00:55:06] Connie Blomgren: There was one paper that came in. And the scholar was located in Ethiopia. And in all the time that I've been at IRRODL, I think there's only been, probably only just that one. And I just-- I thought it was a very important paper from the point of view of representing different educators throughout the world because as we know, most educational research comes out of the USA. And it really dominates how people think about education. [00:55:44] So I think that role of diversifying research and having it, available to a, high impact journal is really a good example of being global, right? Thinking really in that more what's happening in that international, global sphere. So I wouldn't say, from the point of view of research per se, was it. [00:56:10] Riveting. I couldn't say that it was, but I just felt that it was really interesting to see that article eventually get published. Again, representing educators who care about their students wherever they happen to be in the world. I really do believe that educators are interested in the student learning. [00:56:32] However that evidence comes out one way or the other. But, I think that international opportunity of learning from others in other locations is really an interesting part of what our journal offers and is therefore part of that open education, global, openness, environment. [00:56:52] Adnan Qayyum: I had an article recently which excited me. And I feel like I'm pitching underhanded to Aga here 'cause it's not an area which I look at very much, which is, the over digitization of our lives. And this was actually about what that means for students in terms of their engagement with their learning when they're so highly digitized in their lives, in so many different ways, and their attention is being directed. [00:57:16] And so I was very excited about this. It was related to the idea of wellness and digital burnout and things like that. So to me it's oh, okay, that's a, kind of a novel angle on it. And it very much connected it to what it actually might mean for students behaviors within the classroom kind of setting and interactions, et cetera. [00:57:32] You find this kind of the creativity of the researchers. And this was again, to just piggyback on Connie's point, I'm always happy of these novel topics, but also when it's happening in places one doesn't assume because there's tends to be so much focus on education from certain pockets of the world, right? [00:57:52] And you actually find this kind of innovation-- innovative topics happening elsewhere. The idea of the journals to create the platform for those people to have access to share those stories. [00:58:02] Terry Anderson: I could share an example of a special issue that we did, which called it, "A Bridge over Troubled Waters" 'cause it was trying to look at contributions from the Arab world as well as from the Israeli world. [00:58:17] And so we managed to find an editor from the Open University of Israel and another one from the Open University in Turkey. And, we thought they didn't get a lot of dialogue amongst the authors. The editors did to a certain degree for sure, but it was also an international learning experience for me in the sense that with no slight against our Arab scholars, but the Israeli scholarship was just so much higher because they have a tradition and a value of formal education and university structure that is so much more advanced than what we see in much of the Arab world. And, and I was felt at the same time this was an interesting, issue. It did raise this,-- it made this visible, this, this discontinuity between the, quality of the, of, and the number of, articles too. So, I had hoped that they would be a lot more even handed than it turned out to be. [00:59:24] But in retrospect, maybe that should have expected that kind of thing to happen. [00:59:30] Alan Levine: And that says to me like, it wasn't like this is an idea, let's-- it's,-- it was an experiment. And now as experiments go, sometimes they, do and they don't. So thank you for sharing that one. [00:59:43] Again, I want to respect time and all this, and thank you for this. And, thinking of that idea of the web conferencing, we've tried with our,\ podcast, we always invite people to follow up in our online space to have conversations with us. [00:59:59] It rarely happens, but I'm, coming up with this new thing I call the "Aftercast". And so I just want to ask if anybody's willing to either put a question out there or a provocation or a call to be more involved with IRRIODL that you want to have come through as audio \--. You could go wild and say when I'm doing this podcast in 2050, what are we gonna be talking about for the 50th anniversary of IRRODL? [01:00:29] So dream big? or just what would you like to, hear from, people about IRRODL sure. Future Experiments and Innovations for IRRODL --- [01:00:38] Adnan Qayyum: One of the interesting that emerged me listening to the folks here who've been involved with the hurdle more than Me is I was, I'm really amazed and impressed by the experimentation, which has always happened with IRRODL. I've been involved with other journals, like I said, and they weren't nearly as open to some of the things that Terry and Ana and Connie mentioned about trying. [01:00:56] So I guess my question, or for potential listeners is what are sort of the experiments you'd think would be interesting for our IRRODL to try going forward? What are some of the other, innovations they may or not may not succeed, so they may or may not succeed, but might be worthwhile for us to try. [01:01:14] Aga Palalas: Absolutely. And along these lines, I have to throw in the AI term, notion. How can we have this conversation not mentioning AI and asking people, maybe the question that we have been kicking around and, as a team are going to continue conversation when we meet together face-to-face, in a month from now. [01:01:39] And that is around, balancing innovation and ethical use of AI. In submissions to a journal as well as, how should we be using as reviewers and publishers? How should would you expect us to use AI in an innovative, at the same time, ethical way? So many questions, sub-questions in that umbrella one. [01:02:16] Terry Anderson: I think a question I'd, throw out to listeners is just how do you feel when you get a request for-- to review an IRRODL article? What, do -- are you depressed? Are you flattered? Are you anxious? And do you review for IRRODL when you wouldn't review for other journals? Or how do you decide what/ how you spend your valuable time, when that request comes along? [01:02:45] 'Cause it is a challenge for us to get good reviewers. So the more we learn about it, the better we'll be. [01:02:53] Alan Levine: You played well with my experimental game. This is the first time I've rolled this into the podcast recording. [01:02:58] And just so brilliant again, to have a better understanding about what a-- not just to say "yeah, we know it's successful", but there's so many factors in here that that, make IRRODL successful, the experimentation that Aga mentioned and, the ongoing effort to make IRRODLE like vital and that it means something. [01:03:19] And also like thinking about GenAI and everything else, like it's gotta have some questions about what it means for the world of publishing and research publishing. If we want better summaries and we have summaries of summaries, like somewhere someone has to be doing the writing and so it's, gonna be interesting times. All around and just so happy and thank you Connie, for, bringing this. Great to meet you. Iga and Terry. always good-- and you have, I have to say, Terry, you've got like the radio voice, like I need to emulate that and great to meet you, Adnan. [01:03:53] I have, a feeling we crossed paths 'cause I used to do some work at Penn Penn State. [01:03:57] Adnan Qayyum: You mentioned Saskatchewan. Where's Saskatchewan? I'm from Saskatchewan. That's where [01:04:00] Alan Levine: Oh, okay. I'm outside of Moose Jaw. I'm a little bit west of Moose Jaw. Yeah. [01:04:04] Adnan Qayyum: I grew up in Regina. [01:04:05] Alan Levine: Yeah. Okay. Yeah. Great place in the world. We know this. and [01:04:09] Uh, yeah. Closing Remarks and Acknowledgements --- [01:04:11] Alan Levine: thank you everybody who's listening to this episode of OE Global Voices. [01:04:14] This is what we do here at Open Education Global. And, for each episode, I go away from having a standard music track. So I pick a different music track from the Free Music Archive 'cause it's all licensed Creative Commons. And I found a track there called Anniversary by an artist named Michael Chapman. [01:04:32] It's licensed under an Attribution, Non-commercial license. And you'll have to wait until this gets published to hear the music I pick, but you, I think you'll like it. And then of course, you'll find this episode at, voices dot oh global.org. and we have this new thing called the "After Cast", and we'll see how that flies. [01:04:50] For anybody else, certainly, we urge you, if you're not familiar with the IRRODL, to not only make use of reading and making use of the rich archive, but, we hear that they need reviewers and they, always want to have like papers. And thank you so much for, bringing us the IRRODL story and congratulations. [01:05:11] I don't hear any resting on laurels for 25, so IRRODIL'S going full speed into its 26th year. [01:05:20] Terry Anderson: Thank you for the opportunity, Thanks al. [01:05:22] Adnan Qayyum: Thanks Alan [01:05:22] Aga Palalas: Thank you so very much, Alan. [01:05:24] Connie Blomgren: Thanks. Thank you so much, Alan, It's been great. [01:05:27] Alan Levine: It's time to thank you again, listeners, for tuning into this episode of OE Global Voices, the podcast we do here at Open Education Global. And it was a real thrilling episode to be hearing from the people involved with now and part of the history of IRRODL, the International Review of Research and Distributed Online Learning, celebrating its 25th anniversary. [01:05:51] That is, again, a huge milestone. [01:05:54] Each episode of OE Global Voices features a different musical intro track that I select from the Free Music Archive. We love the Free Music Archive because all their music is openly licensed. For today's show, I found a track called Anniversary by an artist named Michael Chapman, and it's licensed under a Creative Commons attribution, Non-commercial, No derivative works license. [01:06:17] You could find this episode at our site, voices.oeglobal.org, and please be a part of this new feature the "After Cast". You heard some questions recorded during the show. Join us for the after cast conversations in OEG Connect or wherever you interact with us in social media. [01:06:34] If you're listening and would like to share your own open educational work or suggest some future guests, please let us know via our website, or send an email to us voices@oeglobal.org. [01:06:45] We'll be back again soon with more voices from open educators around the world.