Research Article | Open Access | Download Full Text
Volume 1 | Issue 1 | Year 2023 | Article Id: AIR-V1I1P102 DOI: https://doi.org/10.59232/AIR-V1I1P102
Enhancing Institutional Assessment and Reporting Through Conversational Technologies: Exploring the Potential of AI-Powered Tools and Natural Language Processing
James Hutson, Dan Plate
| Received | Revised | Accepted | Published |
|---|---|---|---|
| 07 Apr 2023 | 12 Jun 2023 | 27 Jun 2023 | 07 Jul 2023 |
Citation
James Hutson, Dan Plate. “Enhancing Institutional Assessment and Reporting Through Conversational Technologies: Exploring the Potential of AI-Powered Tools and Natural Language Processing.” DS Journal of Artificial Intelligence and Robotics, vol. 1, no. 1, pp. 11-22, 2023.
Abstract
Keywords
Human-machine interaction, Conversational AI, AI application, AI technique, AI-assisted mathematics, Natural language processing, Recognition and applications
References
[1] David Carless, How Chatgpt Can Help Disrupt Assessment Overload, Times Higher Education, 2023. [Online]. Available: https://www.timeshighereducation.com/campus/how-chatgpt-can-help-disrupt-assessment-overload
[2] Ainia Hidayah, and Syahrani Syahrani, “Internal Quality Assurance System of Education in Financing Standards and Assessment Standards,” Indonesian Journal of Education, vol. 2, no. 3, pp. 291-300, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Jandhyala B. G. Tilak, and Amruth G. Kumar, “Policy Changes in Global Higher Education: What Lessons Do We Learn From the COVID-19 Pandemic?,” Higher Education Policy, vol. 35, no. 3, pp. 610-28, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Natasha Quadlin, and Brian Powell, Who Should Pay?: Higher Education, Responsibility, and the Public, Russell Sage Foundation, 2022.
[5] Krystal Hu, Chatgpt Sets Record for Fastest-Growing User Base - Analyst Note, Reuters. [Online]. Available: https://www.reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/
[6] Jakob Nielsen, Chatgpt Lifts Business Professionals’ Productivity and Improves Work Quality, Neilsen Norman Group. [Online]. Available: https://www.nngroup.com/articles/chatgpt-productivity/
[7] ANTHROP/C, Introducing 100K Context Windows. [Online]. Available: https://www.anthropic.com/index/100k-context-windows
[8] Partha Pratim Ray, “Chatgpt: A Comprehensive Review on Background, Applications, Key Challenges, Bias, Ethics, Limitations and Future Scope,” Internet of Things and Cyber-Physical Systems, vol. 3, pp. 121-154, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[9] Nathaniel Prottas, “The Power of Words,” Journal of Museum Education, vol. 48, no. 1, pp. 1-6, 2023.
[CrossRef] [Publisher Link]
[10] Teo Susnjak, Gomathy Suganya Ramaswami, and Anuradha Mathrani, “Learning Analytics Dashboard: A Tool for Providing Actionable Insights to Learners,” International Journal of Educational Technology in Higher Education, vol. 19, no. 1, p. 12, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[11] Donnie Adams, and Kee-Man Chuah, “Artificial Intelligence-Based Tools in Research Writing: Current Trends and Future Potentials,” Artificial Intelligence in Higher Education, pp. 169-84, 2022.
[Google Scholar] [Publisher Link]
[12] Harold Goss, “Student Learning Outcomes Assessment in Higher Education and in Academic Libraries: A Review of the Literature,” The Journal of Academic Librarianship, vol. 48, no. 2, p. 102485, 2022. [CrossRef] [Google Scholar] [Publisher Link]
[13] Anca Antoaneta Vărzaru, “Assessing Artificial Intelligence Technology Acceptance in Managerial Accounting,” Electronics, vol. 11, no. 14, p. 2256, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[14] Yogesh K. Dwivedi et al., “So What If Chatgpt Wrote It?” Multidisciplinary Perspectives on Opportunities, Challenges and Implications of Generative Conversational AI for Research, Practice and Policy,” International Journal of Information Management, vol. 71, p. 102642, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[15] Sergio Salinas Monroy et al., “Blockchain-Empowered Distributed Additive Manufacturing-As-A-Service: An Architectural Perspective,” IEEE Network, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[16] Walid Ibrahim et al., “An Online Management System for Streamlining and Enhancing the Quality of Learning Outcomes Assessment,” Education and Information Technologies, vol. 27, no. 8, pp. 11325-11353, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[17] Ingrid Novodvorsky et al., “Supporting and Evaluating Learning Outcomes Assessment through Academic Program Reviews,” Assessment Update, vol. 27, no. 4, pp. 4-16, 2015.
[CrossRef] [Google Scholar] [Publisher Link]
[18] Matthew Sowcik, Jill L. Lindsey, and David M. Rosch, “A Collective Effort to Understand Formalized Program Review,” Journal of Leadership Studies, vol. 6, no. 3, pp. 67-72, 2012.
[CrossRef] [Google Scholar] [Publisher Link]
[19] Kerry Wu, and Heidi Senior, “Business Librarians and New Academic Program Review,” Journal of Business & Finance Librarianship, vol. 21, no. 2, pp. 114-134, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[20] V. Coombs, Institutions Should Link Program Reviews to Strategic Plans, Inside Higher Ed, 2017. [Online]. Available: https://www.insidehighered.com/blogs/call-action-marketing-andcommunications-higher-education/institutions-should-link-program
[21] Lynne Bowker, “Language and Quality Assurance: A Case Study Highlighting the Effects of Power, Resistance, and Countertactics in Academic Program Reviews,” TTR, vol. 29, no. 2, pp. 177-193, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[22] Kevin Dougherty, and Vikash Reddy, “Performance Funding for Higher Education: What Are the Mechanisms? What Are the Impacts?,” ASHE Higher Education Report, vol. 39, no. 2, 2013.
[23] Ingrid Novodvorsky, Elaine Marchello, and Lisa Elfring, “Assessment Work in an Academic Professional Development Center,” The Palgrave Handbook of Academic Professional Development Centers, Cham: Springer International Publishing, pp. 97-108, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[24] Ashley L. Siems, and Michael Bell, “Utilizing Specialty in Training Examinations and Annual Program Review to Improve Program Curriculum,” Academic Pediatrics, vol. 18, no. 5, p. E16, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[25] Thomas L. Harnisch, “Performance-Based Funding: A Re-Emerging Strategy in Public Higher Education Financing (Policy Brief),” American Association of State Colleges and Universities, 2011.
[Google Scholar] [Publisher Link]
[26] Robert C. Dickeson, Prioritizing Academic Programs and Services: Reallocating Resources to Achieve Strategic Balance, John Wiley & Sons, 2010.
[Google Scholar] [Publisher Link]
[27] David J. Deming, and David Figlio, “Accountability in US Education: Applying Lessons from K-12 Experience to Higher Education,” Journal of Economic Perspectives, vol. 30, no. 3, pp. 33-56, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[28] Higher Learning Commission, (N.D.), Quality Initiative. [Online]. Available: https://www.hlcommission.org/accreditation/quality-initiative.html
[29] A. Permenter, Academic Program Review: From Critical Processes to Implementation. Southern Association of Colleges and Schools Commission on Colleges, 121st Annual Meeting, 2016. [Online]. Available: http://www.sacscoc.org/2016amho/cs%20handouts/cs-62_permenter.pdf
[30] George D. Kuh et al., Student Success in College: Creating Conditions That Matter, John Wiley & Sons, 2011.
[Google Scholar] [Publisher Link]
[31] Brenda M. Coppard et al., Program Review: Leveraging Structure, Process and Virtual Reality, 2015.
[Google Scholar] [Publisher Link]
[32] Drexel University, (N.D.) Academic Program Review. [Online]. Available: https://drexel.edu/pheal/assessmenteffectiveness/academic-review/
[33] Elinor Nugent, and Susan Larocco, “Comprehensive Review of an Accelerated Nursing Program: A Quality Improvement Project,” Dimensions of Critical Care Nursing, vol. 33, no. 4, pp. 226-233, 2014.
[CrossRef] [Google Scholar] [Publisher Link]
[34] University of Oklahoma, Academic Program Review, Chapter 3 – Academic Affairs, 2009. [Online]. Available: https://www.okhighered.org/state-system/policy-procedures/chapter%203-%202010%20final3.pdf#page=74
[35] Phil Backlund et al., “NCA Program Review Standards: Background, Application, and Data,” Communication Education, vol. 60, no. 3, pp. 279-95, 2011.
[CrossRef] [Google Scholar] [Publisher Link]
[36] Jane S. Halonen, and Dana S. Dunn, “Avoiding the Potholes of Program Review,” The Chronicle of Higher Education, 2017.
[Google Scholar] [Publisher Link]
[37] M. Morriss-Olson, Feasibility Checklist: the Science of Bringing New Academic Programs to Life, Academic Impressions, 2016. [Online]. Available: https://www.academicimpressions.com
[38] University of Illinois at Urbana-Champaign, (N.D.), Academic Program Review. [Online]. Available: https://provost.illinois.edu/assessment/academic-program-review/#sthash.c04mu9za.dpbs
[39] Lindsay N. Turner, “Quality Assurance in Online Graduate Education: Program Review Processes and Assessment Techniques Used in Higher Education,” Graduate Theses and Dissertations, 2016.
[Google Scholar] [Publisher Link]
[40] Karen D. Mattingly, Margaret C. Rice, and Zane L. Berge, “Learning Analytics as a Tool for Closing the Assessment Loop in Higher Education,” Knowledge Management & E-Learning: An International Journal, vol. 4, no. 3, 2012.
[CrossRef] [Google Scholar] [Publisher Link]
[41] Samiya Khan, Kashish Ara Shakil, and Mansaf Alam, “Big Data Computing Using Cloud-Based Technologies, Challenges and Future Perspectives,” Arxiv Preprint Arxiv:1712.05233, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[42] Julius Murumba, and Elyjoy Micheni, “Big Data Analytics in Higher Education: A Review,” The International Journal of Engineering and Science, vol 6, no. 6, pp. 14-21, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[43] T. Ananthi Claral Mary, and P. J. Arul Leena Rose, “The Impact of Graduate Student’s Perceptions Towards the Usage of Cloud Computing in Higher Education Sectors,” Universal Journal of Educational Research, vol. 8, no. 11, pp. 5463-5478, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[44] Isitor Emmanuel, and Clare Stanier, “Defining Big Data,” Proceedings of the International Conference on Big Data and Advanced Wireless Technologies, pp. 1-6, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[45] Sayantan Khanra et al., “Big Data Analytics in Healthcare: A Systematic Literature Review,” Enterprise Information Systems, vol. 14, no. 7, pp. 878-912, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[46] Glenn Rosenbluth et al., “Variation in Printed Handoff Documents: Results and Recommendations from a Multicenter Needs Assessment,” Journal of Hospital Medicine, vol. 10, no. 8, pp. 517-524, 2015.
[CrossRef] [Google Scholar] [Publisher Link]
[47] Mohamed Azlan Ashaari et al., “Big Data Analytics Capability for Improved Performance of Higher Education Institutions in the Era of IR 4.0: A Multi-Analytical SEM & ANN Perspective,” Technological Forecasting and Social Change, vol. 173, p. 121119, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[48] Yuhuan Cui et al., “A Survey on Big Data-Enabled Innovative Online Education Systems During the COVID-19 Pandemic,” Journal of Innovation & Knowledge, vol. 8, no. 1, p. 100295, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[49] Tawseef Ahmad Naqishbandi, and N. Ayyanathan, “Clinical Big Data Predictive Analytics Transforming Healthcare:-An Integrated Framework for Promise Towards Value-Based Healthcare,” Advances in Decision Sciences, Image Processing, Security and Computer Vision: International Conference on Emerging Trends in Engineering, Springer International Publishing, vol. 2, pp. 545-561, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[50] Ashraf Alam, “Cloud-Based E-Learning: Scaffolding the Environment for Adaptive E-Learning Ecosystem Based on Cloud Computing Infrastructure,” Computer Communication, Networking and Iot: Proceedings of 5th ICICC 2021, Singapore: Springer Nature Singapore, vol. 2, pp. 1-9, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[51] Brooke Hansen, Peter Stiling, and Whitney Fung Uy, “Innovations and Challenges in SDG Integration and Reporting in Higher Education: A Case Study From the University of South Florida,” International Journal of Sustainability in Higher Education, vol. 22, no. 5, pp. 1002-1021, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[52] Brent D. Ruben, and Ralph A. Gigliotti, “The Excellence in Higher Education Model: A Baldrige‐Based Tool for Organizational Assessment and Improvement for Colleges and Universities,” Global Business and Organizational Excellence, vol. 38, no, 4, pp. 26-37, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[53] Kathryn Parker Boudett, and Elizabeth A. City, Meeting Wise: Making the Most of Collaborative Time for Educators, Harvard Education Press, 2014.
[Google Scholar] [Publisher Link]
[54] Huong Thi Pham, “Impacts of Higher Education Quality Accreditation: A Case Study in Vietnam,” Quality in Higher Education, vol. 24, no. 2, pp. 168-185, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[55] Randy Elliot Bennett, “Formative Assessment: A Critical Review,” Assessment in Education: Principles, Policy & Practice, vol. 18, no. 1, pp. 5-25, 2011.
[CrossRef] [Google Scholar] [Publisher Link]
[56] George D. Kuh, “Assessing What Really Matters to Student Learning Inside the National Survey of Student Engagement,” Change: The Magazine of Higher Learning, vol. 33, no. 3, pp. 10-17, 2001.
[CrossRef] [Google Scholar] [Publisher Link]
[57] Florence Martin et al., “Award-Winning Faculty Online Teaching Practices: Course Design, Assessment and Evaluation, and Facilitation,” The Internet and Higher Education, vol. 42, pp. 34-43, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[58] Lorenz S Neuwirth, Svetlana Jovic, and B Runi Mukherji, “Reimagining Higher Education During and Post-COVID-19: Challenges and Opportunities,” Journal of Adult and Continuing Education, vol. 27, no. 2, pp. 141-156, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[59] Victor Manuel López Pastor, “Best Practices in Academic Assessment in Higher Education: A Case in Formative and Shared Assessment,” Journal of Technology and Science Education, vol. 1, no. 2, pp. 25-39, 2011.
[CrossRef] [Google Scholar] [Publisher Link]
[60] Naomi Winstone, and David Carless, Designing Effective Feedback Processes in Higher Education: A Learning-Focused Approach, Routledge, 2019.
[Google Scholar] [Publisher Link]
[61] Nishita Mehta, Anil Pandit, and Sharvari Shukla, “Transforming Healthcare with Big Data Analytics and Artificial Intelligence: A Systematic Mapping Study,” Journal of Biomedical Informatics, vol. 100, p. 103311, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[62] Sebastian Wollny et al., “Are We There Yet?-A Systematic Literature Review on Chatbots in Education,” Frontiers in Artificial Intelligence, vol. 4, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[63] Mohammad Awad AlAfnan et al., “Chatgpt as an Educational Tool: Opportunities, Challenges, and Recommendations for Communication, Business Writing, and Composition Courses,” Journal of Artificial Intelligence and Technology, vol. 3, no. 2, pp. 60-68, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[64] Jie Sheng et al., “COVID‐19 Pandemic in the New Era of Big Data Analytics: Methodological Innovations and Future Research Directions,” British Journal of Management, vol. 32, no. 4, pp. 1164-1183, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[65] Ozgur Ozdemir et al., “A Critical Reflection on Digitalization for the Hospitality and Tourism Industry: Value Implications for Stakeholders,” International Journal of Contemporary Hospitality Management, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[66] Enkelejda Kasneci et al., “Chatgpt for Good? On Opportunities and Challenges of Large Language Models for Education,” Learning and Individual Differences, vol. 103, p. 102274, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[67] Gunther Eysenbach, “The Role of Chatgpt, Generative Language Models, and Artificial Intelligence in Medical Education: A Conversation with Chatgpt and a Call for Papers,” JMIR Medical Education, vol. 9, no. 1, p. e46885, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[68] Víctor González-Calatayud, Paz Prendes-Espinosa, and Rosabel Roig-Vila, “Artificial Intelligence for Student Assessment: A Systematic Review,” Applied Sciences, vol. 11, no. 12, p. 5467, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[69] Zohar Efroni et al., “Privacy Icons: A Risk-Based Approach to Visualisation of Data Processing,” European Data Protection Law Review, vol. 5, p. 352, 2019.
[Google Scholar] [Publisher Link]
[70] Stuart G. Higgins, Akemi A. Nogiwa-Valdez, and Molly M. Stevens, “Considerations for Implementing Electronic Laboratory Notebooks in an Academic Research Environment,” Nature Protocols, vol. 17, no. 2, pp. 179-189, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[71] Sonia Jawaid Shaikh, “Artificially Intelligent, Interactive, and Assistive Machines: A Definitional Framework for Intelligent Assistants,” International Journal of Human–Computer Interaction, vol. 39, no. 4, pp. 776-789, 2023.