Strategies for collaboratively maintaining A- Z lists Authors: Hannah McKelvey & Star Bradley This is an Accepted Manuscript of an article published by Taylor & Francis in Journal of Electronic Resources Librarianship on October 2, 2021, available online: http:// www.tandfonline.com/doi/full/10.1080/1941126X.2021.1988442. McKelvey, Hannah, and Star Bradley. “Strategies for Collaboratively Maintaining A-Z Lists.” Journal of Electronic Resources Librarianship 33, no. 4 (October 2, 2021): 239–251. doi:10.1080/1941126x.2021.1988442. Made available through Montana State University’s ScholarWorks scholarworks.montana.edu Strategies for Collaboratively Maintaining A-Z Lists Hannah McKelvey, Electronic Resources & Discovery Services Librarian, Montana State University Library Star Bradley, Research & Instruction Librarian, Montana State University Library 1 Abstract: This article discusses a project to integrate the management expertise of an Electronic Resources Librarian with the experiences of users, including students and Research & Instruction Librarians, to create a more user-centered A-Z database list. The authors analyzed the database lists of 10 peer institutions that use the Springshare A-Z Database List, gathered data about their own database list, surveyed colleagues, and developed an assignment for two library classes to understand how students and librarians use database lists. The article summarizes their findings and includes recommendations for collaboratively maintaining database lists that can be implemented at any library. Keywords a-z lists; collaboration; electronic resource management; library instruction; user experience 2 Introduction First launched in 2009, Springshare’s A-Z Database List is a popular tool used by many libraries around the globe to manage their database holdings (Springshare, 2009). Since then, there has not been much research that discusses how this Springshare product, or A-Z database lists in general, are utilized and managed by different libraries. Quinn (2020) noted that while “most libraries have a database A–Z list to help patrons find specific resources… there are unfortunately few papers in the professional literature on managing A–Z lists” (p. 132). Hoeppner (2017) does offer some practical tips and advice for managing an A-Z List, including developing selection criteria, and noting the granularity of entry links while acknowledging that most of the published literature does not discuss strategies for managing database lists. Other published research on this topic is focused on the usability of A-Z lists. In 2017, McLaughlin & Bergart released an internal report about how their students experience the A-Z database list, and their research concluded that students are skeptical about “Best Bets” as they can appear to be sponsored ads similar to what might be found during an open web search. A second study by Brisbin and Parlette-Stewart also noted that students ignored “Best Bet” resources due to the default yellow box surrounding them as it was too ad-like (Rubel, 2018). Another case study at the University of North Carolina Greensboro’s University Library demonstrated that students have a hard time locating subject resources from a library homepage which led to multiple improvements and changes to their A-Z list (Hill, 2020). On the other hand, Springshare’s A-Z List LibGuides counterpart is written about quite extensively, including best practices, customizations, accessibility, and using LibGuides as instructional tools. (Gonzalez & Westbrock, 2010; Conerton & Goldenstein, 2017; Pionke & Manson, 2018). 3 To address this gap on the topic of managing A–Z lists, in summer 2020, two librarians at Montana State University (MSU) Library developed a project to integrate the management expertise of the Electronic Resources Librarian with the experiences of library users, including students and Research & Instruction Librarians to create a more user-friendly and user-centered A-Z database list. Ultimately, their goal is to maintain a useful A-Z list through collaboration which is critical to achieving this goal—internal collaboration within a library can greatly improve services and bringing users into that process is beneficial (Atkinson, 2019). This article offers best practices for collaboratively managing and maintaining database lists that can be implemented at any library. Project Overview This project officially began in August 2020, and the key components of the project included analyzing the A-Z database list of peer institutions that were also using the Springshare product; analyzing the MSU Library A-Z database list to gain a better understanding of how it was being used by both students and librarians; gathering feedback directly from Research & Instruction Librarians; and developing assignments for students in credit bearing courses taught by librarians that focused on student use of the A-Z database list. The project originated for several different reasons including lack of policies, limited understanding of usage, and instructional needs. The Electronic Resources Librarian regularly receives questions regarding why certain resources are included on the A-Z database list. However, without an official policy, it’s hard to field these requests clearly and consistently. By gaining a better understanding of how the A-Z list is used at their institution through this project, the authors hoped to develop a policy for what is added to the database list. Secondly, they rarely looked at the usage clicks of these resources. Having used this product for nearly five years, this 4 project was an opportunity to review the list for relevancy and to freshen it up. Lastly, because this is a primary tool that instruction librarians use, they wanted to make sure it was meeting their needs and expectations. Peer Analysis The author's work began with looking at the A-Z Lists of their peer institutions that were also using the Springshare product to try and determine what a typical A-Z Database List looks like. Using their university system’s peer analysis, they found that ten of 14 peer institutions were using the Springshare A-Z Database List. For the analysis, they went to each of the ten institutions' websites between August 26, 2020 - September 17, 2020 to collect the number of databases, database types, vendors, subjects, and best bets per subject at each institution. They also looked at the number of LibGuides tied to each subject per institution. Combined with their own A-Z list data, they found the following: 9% of libraries had less than 200 Databases; 27% of libraries had 201-300 databases; 36% of libraries had 301-500 databases; and 18% of libraries had more than 501 database entries (Figure 1). 5 There were 366 unique subject types across 11 institutions. After normalizing and grouping subjects that were similar, there were a total of 217 unique database types. The following subjects had the most Best Bet associations across all institutions: Nursing; Business; Engineering; English; and Anthropology. The average number of databases that are tagged as Best Bets across the 11 institutions is 139 (or 49% of all databases). Only one institution did not have any databases tagged as Best Bets. Another had 87% of all their A-Z list entries tagged as Best Bets. “History” was the most widely used subject type across 11 institutions, followed by “Business”, “Engineering”, “Political Science”, and “Biology”; “Distance Education” was the least used subject. “Business” had the most associated LibGuides across institutions, followed by “History”, and “Education”. 36% of institutions had less than 50 subjects; 54% of institutions had 51-99 subjects; and only 9% of institutions had more than 100 subjects (Figure 2). 6 Two of the 11 institutions did not use any database types, defined by Springshare as a way to “help easily identify your full-text databases from your abstract databases, and your eBook platforms from your streaming video platforms,” (n.d.a). 45% of institutions had between 1-10 database types; 36% of institutions had between 11-20 database types, while 18% of institutions had 20 or more database types (Figure 3). The most widely used database types across institutions were Articles; Database, Primary Source; Encyclopedias, Dictionaries & Handbooks; Full Text; and Abstracts/Indexes. The authors noted that at their institution alone, over half (54%) of the databases had Article as a subject association. The least used database types were Abstracts; Company Financials; Plays; Statistics; Tests & Measurements; and University Archives—each of these had only one association across all institutions. There was also not one single database type that was in use across all institutions. 7 They also noted that most institutions were using all the features of the Springshare A-Z Database List including New and Trial tags, the Popular tag, Best Bets, Subjects, Database Types, All Vendors/Providers, and the Search for Databases option. Most institutions also used the default name of the list: A-Z Databases. Local Analysis There are several levels of statistics that can be accessed from within Springshare for the A-Z Databases page including page views and click statistics for individual database entries (Springshare, n.d.b). For the authors local analysis, they used these options, the Analyze Holdings feature which Springshare is no longer supporting (Springshare, 2021), and manual data retrieval. At the time of this analysis (March 2020), the authors A-Z list consisted of over 300 active resources, 46 unique subjects, and 23 unique vendors. They started their analysis by getting a general sense of use of the A-Z list between January 1, 2016 - December 31, 2020. 8 What they found is that usage peaked in 2018 before declining in 2019 and again, in 2020 (Figure 4). They also looked at the top five most used databases in this same timeframe, and they found that the same seven databases had most of the use year after year (Figure 5). A-Z List between January 1, 2016 - December 31, 2020. 9 January 1, 2016 - December 31, 2020. To understand if there were any correlations among these seven resources, the authors looked at how many subjects they were associated with, how many locations each was mapped to, and how many, if any, were tagged as best bets (Figure 6). CINAHL Complete, the most used database (21.3% of all click usage), is associated with two subjects, it is mapped in 18 locations, and is a best bet for one subject (Nursing & Medicine). All the remaining databases, apart from Qualtrics, also have associated subjects and are tagged as a best bet for at least one subject. 10 five most used databases between January 1, 2016 - December 31, 2020. They also did this same analysis on the ten resources that had five or less usage clicks. Seven of these resources have zero 0 subject associations, only four of them are mapped to a LibGuide, and none of them are tagged as Best Bets (Figure 7). These findings have pushed the authors to look at these specific resources to see if there are existing subjects that can be associated with them or if any should become best bets. One of their main observations is that five of these low usage resources are local digital collections that might benefit from being merged into a single resource entry. 11 used databases between January 1, 2016 - December 31, 2020. They also analyzed the number of databases and Best Bets by subject (Figure 8). This specific analysis was to draw attention to any subjects that might benefit from having more or less databases associated with it, or if an area is completely lacking Best Bets. At the time of this analysis, the authors found that “History” had an overwhelming number of databases associated with it (90). This finding directed them to work with their humanities librarian to determine if all databases associated with “History” were relevant. 12 They also looked at individual subject areas to see how Best Bets by subject perform. Figure 9 shows the “Nursing and Medicine” Best Bet clicks between January 1, 2020 - December 31, 2020. This figure shows the Best Bets in alphabetical order, just as they would appear on the page to a user. Before they gathered the click usage data, the authors thought that the databases that appear above the fold of the computer monitor would be the ones with the most use. However, their findings show that is not true and that the resources below the fold have the most use of all the Best Bets except for CINAHL Complete. Non-guide clicks are considered clicks that come from the A-Z Database List while the ‘all clicks’ count is the count of clicks from the A-Z Database List and any associated mappings such as a LibGuides link. They also took the “Nursing and Medicine” Best Bets and evaluated the number of locations that each resource is mapped which includes the A-Z list as a location. Next, they generated the 13 number of clicks by location (Figure 10). To provide an example, the Cochrane Library is mapped in 11 places, but only seven (64%) of those locations have any associated clicks. Lastly, they noted subjects that seemed similar or duplicative. For example, they noted that they had “Library Research” and “Information Literacy” as two unique subjects with the same databases associated with them. This is something that has since been streamlined as a direct result of this project. between January 1, 2020 - December 31, 2020. 14 locations and associated number of clicks with each location. Gathering User Perspectives In addition to collecting quantitative data, the authors also wanted to gather qualitative data from their colleagues around their use of our A-Z Database List. However, due to COVID restrictions in place throughout 2020, they were limited to virtual interactions to gather feedback from users. To comply with restrictions, they created a survey for librarians and two assignments to distribute to students who were enrolled in librarian-taught courses in Spring 2021. Librarian Feedback The authors developed and distributed a very short survey (Appendix A) to their Research and Instruction Librarians. They selected a small, targeted group of users that they considered key stakeholders as “testing with 5 people lets you find almost as many usability 15 problems as you'd find using many more test participants” (Nielsen, 2012). The survey took about 5-10 minutes to complete and focused on navigation, management, and teaching with the A-Z Database List. The goal of this survey was to collect perspectives from this group of users that would give the authors additional insight to use along with the other data that was collected to make informed changes to the A-Z Database List. Some of the questions they included were focused on the use of the features of the A-Z Database List such as what filters the librarians use. The responses to this specific question (Figure 11) show that the “By Subject” filter is heavily used (41.7%) followed by the “Format” filter (33.3%). used the most within the A-Z List. One of the goals of this project is to encourage colleagues to edit the A-Z List and gauge their comfort level with editing the A-Z Database List—most respondents said they would be 16 comfortable doing this. The authors want to empower library faculty to make changes to the A-Z list on their own. This will allow teaching librarians to update parts of the list, such as the Best Bets, to go hand in hand with their instructional efforts. Student Feedback A second goal of this project was to do user experience (UX) testing with students on the usability of the A-Z List. However, due to COVID restrictions the authors were unable to conduct the level of testing that they desired. What they were able to successfully implement was some UX testing with students in two classes being taught by library faculty. The two classes were LSCI-121 (Appendix B), an introductory library research skills class, and LS-350 (Appendix C), an upper division literature review and analysis class. The two instructors for these classes modified an existing assignment to include a section that asked students to complete specific tasks. As part of the assignment students in both classes were asked to first start at the library home page and navigate to a specific database. The students in LS-350 were asked to navigate to ProQuest Central and the students in LSCI-121 were asked to navigate to Access World News. Students were asked to detail the steps taken to reach the required resource. The students in both classes were then asked to start at the A-Z List and choose a database that would assist with their research. The goal was to get feedback on how students navigate to databases on their own and learn how they used the A-Z List once they were there. Of the feedback received from both classes, only one student used a Best Bet and a few students had trouble navigating to the A-Z list from the library’s homepage. One student commented that they “got lost” during the process. An issue that confused several students was a database landing page, which falls between the MSU Library homepage and A-Z list. 17 Overall, the authors found that once students reached the A-Z List they were able to search for databases by title, subject, and format to find what they needed. The only search feature on the A-Z List that was not used by this group was the search box that allows users to search for databases by name. Between both classes only one student mentioned using a Best Bet, and none of the students mentioned using the Good Starting Points or New & Trial Resources boxes. Recommendations Based on the authors findings, they have developed several recommendations that can be implemented by any library to improve their own A-Z Database List: Create a Guiding Statement Create a policy or define what resources will be added to an A-Z List (ERIL-L, 2018; Hoeppner, 2017). This will be helpful for Electronic Resources Librarians that manage A-Z Lists to be consistent and fair with what is added, and it will be a way to communicate to colleagues about what exactly is included in the A-Z List. The authors developed the following criteria to guide what resources are added to their own A-Z List: ● Any databases or resources that have been purchased or licensed by the library will be considered for the A-Z List. ● If a database has several sub-databases within it (for example, ProQuest Central), the Electronic Resources Librarian will collaborate with appropriate stakeholders to determine if sub-databases need their own entry. ● No individual journal or book title purchases or subscriptions will be added unless they have been identified as a significant or major work. 18 ● Best Bets will be determined based on relevancy to a specific subject and current instruction needs. They will be reviewed, removed, and updated regularly. ● Open access or freely available resources will be added that fit within collection development policy guidelines. ● If resources do not meet these criteria, it will be suggested that they are added to a subject LibGuide instead. Collaborate and Review Regularly To maintain a flexible, current A-Z List, it is important to review A-Z Lists regularly and set some guidelines to follow. For example, reviewing click usage bi-yearly or yearly may help to spot under used or broken resources that need to be removed. Providing a list of resources with low usage to the Collection Development Librarian can be a helpful way to identify resources for possible cancellations and be used in conjunction with COUNTER-compliant usage data or other resource provider-supplied usage statistics. If a library has subject librarians or liaisons, meeting regularly or asking them for feedback on subjects, formats, and database associations will encourage active participation and help an A-Z List stay fresh and relevant. Conducting usability studies can also help to understand how a list is being used (McLaughlin & Bergart, 2017; Rubel, 2018). Springshare’s LibWizard tool also offers a great way to conduct virtual assessments and reach a wide variety of users (Narlock & Shelton, 2021). Offer Trainings or Workshops Holding regular training sessions or workshops for colleagues can be effective in ensuring that everyone knows about all the features in the A-Z List. The authors received feedback during their project that showed a need for training. For example, there were some 19 suggestions to add subjects and formats that were already listed in the A-Z List. Additionally, training opportunities are a great way to encourage colleagues to update the A-Z List so that it aligns with current instruction needs, for example, modifying Best Bets. A goal of the authors, as an outcome of this project, is to provide click usage data more regularly to their teaching librarians so that they can modify the A-Z List as necessary. Tidy Up Another easy way to keep an A-Z List tidy is to look at subjects and formats regularly. Are they current? Are there duplicative subjects or formats? Before embarking on this path, note that subjects are tied to LibGuides, so be careful what gets deleted or renamed. The authors findings also show that most entries without any subject or format associations tend to get limited usage. Occasionally, A-Z Lists contain sub-databases that have been broken out from a main interface. Look at the click usage for these types of resources to determine if it might be beneficial, both from a management and user perspective, to merge these into a single resource entry and use the alternative names/keywords or more information fields to include the names of what’s now included in the single entry—this field is searchable in the Springshare A-Z Database instance. “Weeding the list to reduce the number of entries [also makes] a sleek, fast list” (Hoeppner, 2017, p. 36). Lastly, check the assets within the Springshare system for resources that have been linked individually from a LibGuide but that have a database entry. Updating these links to point to the database entry will ensure that click usage is captured, and it will help clean up the asset list. 20 Use Appropriate Language Thinking about the jargon and language used throughout an A-Z List can be a positive way to improve usability. The authors recently renamed their A-Z Database List to A-Z Databases and Online Resources to show that they include other resources and not solely databases, however, it’s not uncommon to also see lists simply called A-Z List or A-Z Databases (this is the default Springshare name). Using plain language to create short, concise database descriptions can greatly improve usability as well (Fletcher & Adcock, 2020). The Plain Writing Act of 2010, while a requirement for government documents to be written clearly, can also be applied to the writing of database descriptions (Plain Language Action and Information Network, n.d.). The authors also changed the default language of ‘Database Type’ to ‘Format’ in their database list to try and clarify what this specific list of terms represents. Most of the default labels in the Springshare A-Z Database List public interface can easily be customized by navigating to the admin interface and going to ‘Look & Feel’ and ‘Language Options’. These changes do not require advanced skills—they simply require typing in the new name(s) of the labels that are being updated. Conclusion While oftentimes the management of A-Z Lists may fall to the Electronic Resources Librarian, the most effective lists are built out of collaborations with colleagues and users. Through their research and analysis, the authors determined that there is not a one-size-fits-all model when it comes to an A-Z List, however, there are best practices and strategies that can be used to improve A-Z List usability and make maintenance easier for A-Z List managers including creating policies, offering trainings, and seeking regular feedback from both internal 21 and external users. A-Z Lists should be thought of as “living” lists, requiring regular maintenance to ensure their effectiveness for users. References Atkinson, J. (2019). Collaboration by academic libraries: What are the benefits, what are the constraints, and what do you need to do to be successful? New Review of Academic Librarianship. 25:1, 1-7, https://doi.org/10.1080/13614533.2019.1575016 Conerton, K. & Goldenstein, C. (2017). Making LibGuides Work: Student Interviews and Usability Tests. Internet Reference Services Quarterly. 22:1, 43-54. https://doi.org/10.1080/10875301.2017.1290002 ERIL-L. (2018). Criteria For eResources on A-Z Databases List. http://lists.eril- l.org/pipermail/eril-l-eril-l.org/2018-March/003406.html Fletcher, L. & Adcock, S. (2020). Reimagining A to Z Database Descriptions to Improve User Experiences: A Database Description Project. 70th Annual meeting Southern Chapter Medical Library Association. https://dc.uthsc.edu/scmla/2020/posters/15/ Gonzalez, A.C. & Westbrock, T. (2010). Reaching Out with LibGuides: Establishing a Working Set of Best Practices. Journal of Library Administration. 50:5-6, 638-656, https://doi.org/10.1080/01930826.2010.488941 Hill, K. (2020). Usability beyond the Home Page: Bringing Usability into the Technical Services Workflow. The Serials Librarian. 78:1-4, 173-180. https://doi.org/10.1080/0361526X.2020.1702857 Hoeppner, A. (2017). Database Lists A to Z: A Practitioner’s Tips and Caveats for Managing Database Lists. The Serials Librarian: From the Printed Page to the Digital Age. 73:1, 27-43, DOI: https://doi.org/10.1080/0361526X.2017.1320779 22 McLaughlin, J. & Bergart, R. (2017). LibGuides & AZ Database List UX Study - Final Report. The Atrium. http://hdl.handle.net/10214/14436 Narlock, M. & Shelton, A. (2021). LibWizarding & The Magic Of Feedback. https://buzz.springshare.com/videohighlights/springshare-learning-lab/libwizard-magic- feedback Nielsen, J. (2012). How Many Test Users in a Usability Study? Nielsen Norman Group. https://www.nngroup.com/articles/how-many-test-users/ Pionke, J. & Manson, J. (2018). Creating Disability LibGuides with Accessibility in Mind. Journal of Web Librarianship. 12:1, 63-79. https://doig.org/10.1080/19322909.2017.1396277 Plain Language Action and Information Network (PLAIN). (n.d.). What is plain language? Plainlanguage.gov. https://www.plainlanguage.gov/about/definitions/ Quinn, T. (2020). Less is More: A Case Study of Consolidating Two Database A–Z Lists for Better Staff and User Experiences. Association for Library Collections & Technical Services. https://doi.org/10.5860/lrts.64n3.131-138 Rubel, D. (2018). Report on the ER&L Session: A Collaborative Approach to Creating and Maintaining a User-Centered A-Z List. NASIG Newsletter: Serials & E-Resources News. https://tigerprints.clemson.edu/cgi/viewcontent.cgi?article=1920&context=nasig Springshare. (2009). A-Z Database Lists in LibGuides – Beta Testers Needed (Serials Solutions Users). https://blog.springshare.com/2009/09/23/a-z-database-lists-in-libguides-beta- testers-needed-serials-solutions-users/ Springshare. (2021). Springshare Lounge: LibGuides February 2021 Release Notes. https://lounge.springshare.com/discussion/351/libguides-february-2021-release-notes 23 Springshare. (n.d.a). A-Z Database List: add, edit, & delete database types. https://ask.springshare.com/libguides/faq/977 Springshare. (n.d.b). Statistics: A-Z Databases Page. https://ask.springshare.com/libguides/faq/1054 24 Appendix A How do you use the Springshare A-Z Database List? This project will integrate the management expertise of the Electronic Resources Librarian with the experiences of library users, including patrons and our colleagues to create a more user- friendly and user-centered database list. The goal of this survey is to gather the perspectives of all of you, who regularly use the A-Z Database List, to help us create a better database list. Please note, the findings of this survey may be shared through public presentations in academic settings, including discipline specific conferences, as well as be submitted for publication in peer-reviewed journals. All responses to this survey will be anonymized. The survey should take about 5-10 minutes to complete. We appreciate your time and participation in this process! Navigating & Finding Databases and the A-Z Database List 1. How do you navigate to the A-Z Database List? 2. Once you are at the A-Z Database List, what filters do you use? ● All Subjects ● Format ● All Vendors/Providers ● Sort Alphabetically ● None 3. Do you ever use the built-in search option in the A-Z List? ● Yes ● No 4. Do you ever search for databases in our discovery layer? ● Yes ● No 25 5. When searching for databases by subject, do you use the best bets that are listed? ● Yes ● No 6. How do you find database details or more information about a particular database? ● Short description listed under each database ● Select the blue 'more information' icon ● Other Management of the A-Z Database List 7. Would you feel comfortable editing the A-Z Database List? ● Yes ● No ● Maybe, with some training 8. Would you be comfortable changing the best bets for a subject area? ● Yes ● No ● Maybe, with some training Teaching with the A-Z Database List 9. How do you teach students to navigate to and use the A-Z Database List? 10. When you teach (either in a classroom or reference setting) the A-Z Database List to students, do you show them: ● Best Bets ● Subject Types ● Formats ● Vendors ● Other 26 Appendix B LSCI-121 Homework Assignment #2 – Searching for Information (Free v. Deep Web) Part 3: Let’s Practice Searching! 1) Find and Navigate to library databases ● Start at the Library’s homepage ● Do NOT use the search box; locate the database Access World News. ● Describe the steps you took to find the database in detail, noting any difficulties along the way. ● Conduct a search in the database and find a newspaper. ● Copy and paste a link to an article from an issue of the newspaper from February 2021. 2) A-Z List Homepage ● Start at the A-Z Database List. ● Choose a database that will give you information on Educational Research. ● Describe the steps you took to find the database in detail, noting any difficulties along the way. ● Conduct a search in the database. ● Copy and paste a link to an article that talks about classroom management. 27 Appendix C LS-350 Search Analysis Assignment PART 2: 1) Find and Navigate to library databases ● Start at the Library’s homepage ● Do NOT use the search box; locate the database ProQuest Central. ● Describe the steps you took to find the database in detail, noting any difficulties along the way. ● Conduct a search in the database on your topic and find one article. ● Copy and paste the citation from ProQuest for your article. 2) A-Z List Homepage ● Start at the A-Z Database List ● Choose a database that will give you information on your topic. ● Describe the steps you took to find the database in detail, noting any difficulties along the way. ● Conduct a search in the database. ● Copy and paste the citation from the database for your article. 28