AI for collections & vendor evaluation

AI in collection development is real, but it is mostly decision support rather than decision-making. Most of what vendors label AI in this space is machine learning and analytics, not generative AI. The distinction matters because the appropriate professional actions differ. Ex Libris Rialto, the marketplace and selection platform now under Clarivate, embeds AI-driven recommendations that refine search results for selectors based on past purchases and patron engagement. Clarivate has also announced that the Collecto collection analytics suite is being folded into Rialto to support deselection, retention, and space optimization across the Alma community. For example, a selector using such a platform receives ranked recommendations and usage-based deselection candidates, not a finished selection list. The tool narrows the field of attention but does not make the collection decision. That division of labor is the realistic shape of AI in selection today. Much of the more ambitious language in vendor materials describes roadmap features, not capabilities a selector can use this month. To evaluate any selection or analytics tool honestly, ask the vendor to distinguish what is shipping now from what is planned, because the marketing rarely makes that distinction unprompted.

Weeding and deselection follow the same pattern. Data-driven deselection, through usage statistics, retention candidates, and overlap analysis, predates generative AI by years. The current direction layers machine learning onto that established data rather than replacing the librarian's judgment. For example, a machine learning model trained on a library's prior weeding decisions can pre-screen candidates, so a librarian reviews a focused list rather than the entire collection. But the model reproduces patterns in past decisions; it does not exercise the collection knowledge that justified them. Such a tool is genuinely useful for managing scale. Its recommendations must still be read as a starting point for review, not an instruction to discard. Demand-driven and evidence-based acquisition are also well integrated into these platforms. The 2025 disruption to perpetual ebook purchasing, discussed later in this module, directly affects how those workflows can be configured.

The scholarly evidence supports a cautious posture. Portillo and Carson evaluated general large language models for collection development in the Journal of the Medical Library Association in January 2025. They concluded that the models are not yet reliable as primary tools for collection development due to inaccuracies and hallucinations, but can serve as supplementary tools for analyzing subject coverage and identifying gaps. A general model is therefore appropriate for gap analysis against an existing collection, where the librarian can verify every suggestion. It is inappropriate as a source of titles to buy, where a hallucinated citation can enter an order list unchecked.

A newer wrinkle is that AI has become a collection development problem in its own right. AI-generated books have begun entering library collections through aggregator platforms, as documented by 404 Media in February 2025 and discussed at IFLA the same year. Machine-generated titles appear in catalogs without ever passing through selection, a particular hazard in pay-per-use lending models. These titles are precisely the kind of low-quality material that selection exists to keep out, and they arrive through automated supply chains that bypass the selector entirely. In response, libraries are updating collection development policies to name AI-generated content explicitly. None of these tools or problems changes the core of the work. Deciding what a specific community of users actually needs, and standing behind that decision, remains the judgment of the librarian who knows the collection and the patrons it serves.

Acquisitions and licensing is where the gap between expectation and reality is largest. The infrastructure that tracks open access agreements and the tools that review licenses are mostly automation and metadata matching, not generative AI. Open access agreement tracking runs on the ESAC Registry, the cOAlition S Journal Checker Tool, and OA.Report from the nonprofit OA.Works. These are built on structured data and automated analysis rather than language models. For example, a librarian confirming whether an author qualifies for a transformative agreement consults a registry lookup and a checker tool, not a generative assistant. The reliability of that answer depends on metadata quality, not on any AI summarization. The distinction matters: a librarian who expects an AI assistant to interpret an agreement will be disappointed, while one who understands these as data tools will use them correctly. The most explicitly AI-enabled product in this adjacent space, Copyright Clearance Center's OA Intelligence, uses affiliation-matching technology. It is publisher-facing, however, built for modeling open access offers rather than for library acquisitions.

AI-assisted license and contract review, the application most librarians ask about first, barely exists as a library-specific product. The clearest documented case is a 2025 pilot by an assistant director at the San Diego Law Library, published on the AALL CRIV blog. The pilot trialed Spellbook, a general legal-AI Word add-in, on electronic resource license terms and asked it to flag problematic clauses for a public library. For example, the tool flagged ambiguous authorized-user language that matched issues the library had already identified and negotiated. That both validated the tool's usefulness and revealed its limits, since it surfaced a known problem rather than an unknown one. The result is encouraging and modest at the same time. The librarian concluded there is a definite possibility for future application in reviewing contracts, but that human review and human context will always be warranted. A librarian experimenting with a general AI tool on license terms today should treat it as a second reader that may catch what fatigue misses, never as a substitute for reading the agreement.

The more consequential licensing conversation in 2024 through 2026 is not about AI summarizing agreements but about AI clauses inside the agreements themselves. The ICOLC statement on AI in licensing, the Association of Research Libraries' electronic resource licensing guidance, and consortial work on library-friendly AI terms all address this. They concern how text-and-data-mining and AI-use rights are negotiated into electronic resource licenses, and whether uploading licensed content into a public AI tool violates the license. For example, a librarian who pastes a licensed article into a general chatbot to summarize it may be transmitting licensed content to a third party in violation of the agreement. The AI tool will never raise that licensing question on its own. These risks make the human reading of the contract more important, not less. Judging what the institution may lawfully do with licensed content in an AI workflow remains squarely the job of the librarian who negotiates and administers the license.

The patron-facing AI search products that vendors sell most aggressively share a common architecture, and understanding it is the fastest way to evaluate any of them. Nearly all use retrieval-augmented generation. A natural-language question is converted into a search, a small set of results is retrieved from a proprietary index, and a language model summarizes those results with citations. This architecture grounds answers in licensed content and reduces hallucination, but it does not eliminate it. It also introduces a characteristic limitation: the summary reflects only the handful of sources the system retrieved, not the full literature on the question. For example, a tool that retrieves and summarizes five sources will produce a confident, well-written answer whether or not those are the five a subject expert would have chosen. That confident prose is precisely what makes independent testing necessary, because the fluency of the output has no relation to the completeness of the underlying search.

The products differ mainly in what they search and how they are priced. Ex Libris Primo Research Assistant, released to production in September 2024, is bundled at no additional cost for Primo customers. It converts the question to a Boolean query, retrieves results from the Central Discovery Index, and summarizes the top five with inline citations. It searches metadata and abstracts rather than full text and can surface items the institution does not own. Elsevier's Scopus AI draws only on Scopus-indexed peer-reviewed content and offers summaries, expanded summaries, and concept maps; it is a paid add-on with undisclosed pricing. EBSCO's AI Insights and Natural Language Search, launched in 2025 across EBSCOhost and EBSCO Discovery Service, generate key-point summaries and translate conversational queries into Boolean. JSTOR's AI tool, out of beta and available to participating institutions by mid-2025, is document-scoped: it helps a user assess, summarize, and question a single selected item. Clarivate's Web of Science Research Assistant, a paid add-on launched in September 2024, is grounded only in the Web of Science Core Collection. For example, the same reference question will return different answers across these tools. The reason is not that one model is smarter but that each searches a different index with a different scope. That is the single most important fact to convey to a patron who assumes an AI answer is comprehensive. Independent reviewers, including Choice, the Katina reviewer, and university LibGuide authors, consistently rate these tools as useful starting points that require critical evaluation, not authoritative replacements for search.

Not all vendor AI in this space is patron-facing, and the back-office applications are often the more mature. OCLC has deployed AI for WorldCat de-duplication. In December 2025, it added AI metadata enrichment in WorldShare Record Manager and Connexion that suggests classification and subject headings, with catalogers retaining accept-or-reject control. Ex Libris offers a comparable metadata assistant in Alma. For example, the OCLC de-duplication work was trained on tens of thousands of human-labeled duplicate pairs before it was run at scale. That kind of grounding separates a genuinely useful production tool from a demonstration. To evaluate any of these products honestly, use the method independent reviewers use. Run your own real reference queries through the tool, compare the result against a non-AI search of the same index, and read the cited sources rather than trusting the summary. No vendor demonstration can substitute for testing on your own questions, with your own collection and patrons in mind. Whether a tool serves your users is a professional judgment for the librarian who runs that test, not the vendor who designed the demonstration.

The central financial question with vendor AI is whether a feature is bundled into a subscription you already pay for or sold as a separate add-on. That distinction determines both the cost and the leverage you have. Primo Research Assistant and JSTOR's tool are bundled at no additional charge for existing subscribers. Scopus AI and Web of Science Research Assistant are paid add-ons whose pricing is not publicly disclosed and is negotiated case by case. For example, a library evaluating a bundled tool is deciding only whether to turn a feature on. A library evaluating an add-on is committing new money against an already strained budget, and the rigor of the evaluation should scale accordingly. Opaque add-on pricing is a recurring pattern in this market. It is compounded by the fact that none of these tools currently provide COUNTER usage statistics. The traditional cost-per-use analysis libraries rely on to justify renewals therefore cannot yet be performed on AI add-ons.

The budget context makes this more pointed. Clarivate's Pulse of the Library 2026 report, based on 1,876 library workers surveyed in April and May 2026, found that only about one in six libraries has reached moderate or active AI implementation, and a third are still exploring and evaluating. Budget constraints were cited by 52% of respondents as a concern in adopting or scaling AI, level with lack of expertise. Adoption is moving slowly and money is tight at the same time, which is exactly when an opaque, COUNTER-less add-on deserves the most scrutiny. Within most institutions, these content-linked AI tools fall on the library materials and electronic resources budget rather than on central IT. The library bears the cost directly and should weigh it against the books, databases, and staff time the same money would otherwise buy.

The Clarivate ebook controversy of 2025 is the cautionary tale to teach alongside the numbers. In February 2025, Clarivate announced it would phase out one-time perpetual ebook and print purchases, along with demand-driven and evidence-based acquisition, in favor of subscription access. It framed the shift partly around AI-powered discovery. The international backlash was immediate, including an ICOLC statement and a competing vendor publicly reaffirming perpetual and demand-driven options. For example, within weeks Clarivate apologized for the lack of consultation and extended perpetual-purchase ability across its platforms through June 30, 2026. Under the plan Clarivate announced for that deadline, it stopped selling title-by-title perpetual ebooks and print books directly after June 30, 2026. Libraries keep perpetual access to titles they already own on Ebook Central and can still buy perpetual titles from other publishers and aggregators through the Rialto marketplace. The reversal shows how quickly a vendor's strategic positioning can shift and how much leverage a coordinated library response can have. It is a teaching case in vendor lock-in, the fragility of ownership, and the way AI-era platform strategies intersect with budget and collection control. Formal dollar-figure return-on-investment analyses for AI search add-ons are essentially absent from the literature. The value judgment therefore cannot be outsourced to a published benchmark. Deciding whether a given AI product is worth its cost, against everything else the budget must cover, remains the judgment of the librarian accountable for that budget.

Vendor AI search tools raise privacy considerations that differ from those of the underlying databases. The vendor commitments are broadly reassuring, but they vary enough that they must be read clause by clause rather than trusted by reputation. For Primo Research Assistant, Scopus AI, and Web of Science Research Assistant, vendors claim that user queries are processed in private or walled environments and are not used to train public models. Several state that inputs are used only for the current session and not stored. For example, Clarivate states that Web of Science Research Assistant data is not used to train large language models directly or indirectly. It also states that only the user's query, not publisher content or library-owned materials, is passed to the model. JSTOR retains de-identified conversation logs and caps the retention of data sent to its model providers at thirty days. These commitments differ in their specifics, and the differences are exactly where a privacy review must focus: retention windows, whether queries feed analytics, and whether the tool requires a personal login.

The login requirement is the consideration most specific to these tools and the easiest to overlook. Primo Research Assistant and JSTOR's tool require user authentication, so a patron's queries can in principle be associated with an identified individual. That is a different privacy posture than an anonymous database search. For example, a student researching a sensitive health or legal question through an authenticated AI assistant generates a query log tied to their identity. The same search through an anonymous catalog interface would not, which is a meaningful distinction for reader privacy. These identification questions run directly against the profession's long-standing commitment to the confidentiality of patron research, expressed in the Library Digital Privacy Pledge and the American Library Association's intellectual freedom guidance. Independent commentators have also urged libraries to remain deeply skeptical of AI summarization tools that strip citations from their original context, even while cautiously adopting natural-language discovery. A summary that detaches a claim from its source undermines the patron's ability to evaluate it.

The honest state of this area is that no published, tool-specific privacy audit by a named library privacy officer yet exists for these products. Libraries are currently relying on vendor self-description rather than independent verification. This is a genuine gap and an area to monitor. To act responsibly in the meantime, read each product's privacy policy against your own patron privacy standards before enabling the tool. Tell patrons clearly when an AI feature logs identified queries. Treat the vendor's assurances as claims to verify, not facts to assume. That verification, and the decision about whether a tool's privacy posture is acceptable for a specific community, is a professional responsibility. It cannot be delegated to the vendor whose product is under review.

The privacy considerations above become actionable once they are turned into a fixed set of questions asked of every vendor, every time, and ALA's AI guidance provides exactly that list. Vendors volunteer a reliably flattering subset in demonstrations. Instead of relying on it, the guidance directs libraries to establish, as a condition of procurement, whether and how an AI product handles patron and staff data. A standing checklist does not depend on a librarian remembering the right concern in the moment; it builds the concern into the process. A vendor's difficulty answering any single question is itself a finding worth recording.

Each of these maps to a concrete risk rather than an abstract preference. For example, the default-on question matters because a feature enabled by default has already begun processing patron data by the time a library thinks to evaluate it. That inverts the "opt-in" model the guidance elsewhere insists on. The training question matters because a tool that improves its model on patron searches is extracting patron behavior for the vendor's benefit, a concern this curriculum's labor and privacy discussions both address. The retention and deletion question matters because a vendor that cannot say how long it keeps patron queries, or cannot delete them on request, cannot support the library's own confidentiality obligations, however reassuring its marketing. The exit-rights question matters because data governance does not end when a contract does. A library that has not secured the return or destruction of patron data at contract termination has left that data in the vendor's hands indefinitely.

The practical way to use this checklist is to write it into the procurement process itself. Ask these questions during evaluation and document the answers alongside the pricing and feature comparisons libraries already record. For example, a request for information or a vendor questionnaire that includes these six items produces a written record. It can be compared across competing products and revisited at renewal, which is far more durable than a verbal assurance given in a sales meeting and forgotten by the time it matters. The guidance also treats a vendor's unwillingness or inability to answer as informative in its own right. A vendor who cannot describe its data handling in these terms is either not examining its own product closely or choosing not to disclose. Either reading is a reason for caution. What the checklist cannot do is make the adoption decision. Weighing a tool's documented data practices against its value for a specific patron community, and deciding whether the residual risk is acceptable, remains the professional judgment of the librarians accountable for both the data and the service.

The volume of vendor AI offerings makes a repeatable decision framework more valuable than any single product evaluation. The products will change, but the questions a librarian must ask about them will not. Two published frameworks anchor this well. The Choice rubric for evaluating generative AI resources, from November 2024, screens any tool on four quick questions. Transparency asks whether citations are traceable to real sources. Efficiency asks whether the tool augments rather than reinvents an existing workflow. Critical thinking asks whether it preserves rather than eliminates the user's analysis. Privacy asks whether the policy is acceptable and personal data stays out. For example, applying these four screens to a bundled discovery assistant takes a single testing session and produces a defensible yes-or-no on each dimension. That is far more useful to administration than a general impression. The rubric is especially apt for a community college, because it explicitly asks the evaluator to weigh a tool against a specific student population rather than an abstract ideal user. The ACRL AI Competencies for Academic Library Workers, approved in October 2025, provide the complementary professional backbone. They are organized around ethics, knowledge, evaluation, and application, and are deliberately product-agnostic so they can be adapted as tools change.

The practical decision resolves into three outcomes, and naming them explicitly prevents the common failure of drifting into adoption by default. Adopt a feature when it is bundled, passes the rubric, and addresses a real workflow need; the cost of turning on a tested bundled tool is low and the benefit concrete. Pilot a feature, rather than adopting or declining it outright, when it is promising but unproven locally, or when it is a paid add-on of uncertain value. For example, Empire State University and a University of California San Diego team have used a pilot method that runs the same real reference queries through the AI tool and through a non-AI search. Both are scored with the Choice rubric, producing local evidence instead of reliance on the vendor's claims or another institution's experience. Such a pilot turns an opaque purchasing decision into a measured one. It also generates exactly the documentation administration needs to approve or reject the spend. Decline a feature when it fails the rubric on a dimension that matters, when its privacy posture is unacceptable for the patron community, or when its cost cannot be justified against everything else the budget must cover.

The final element of the framework is continuous monitoring rather than a one-time verdict, because everything in this space dates quickly. The Ithaka S+R Generative AI Product Tracker, public and continuously updated since 2024, maintains a living table of these products with their purchasing models, features, and limitations. It is a more reliable current reference than any feature list a librarian could memorize. For example, a selector who bookmarks the tracker and rechecks it each term will catch a bundled tool moving to paid pricing, or a new privacy disclosure, far sooner than one relying on vendor outreach. Pair that attention with a verify-before-teaching habit: reconfirm pricing, privacy clauses, and feature scope each term, since a claim accurate last semester may not be accurate now. To keep this from becoming overwhelming, evaluate only the tools your library actually encounters, apply the same four-screen rubric every time, and document the decision. A disciplined, repeatable process lets a library keep pace with a flood of vendor AI products. At every step, deciding which products earn a place in the collection and the budget remains the responsibility of the librarians who know their patrons and resources best.

Every framework in this module so far assumes the library is making a decision: evaluating a product, weighing a cost, or choosing to adopt, pilot, or decline. A substantial and growing share of the AI a library actually runs never passed through that decision. It arrived inside a database the library already subscribed to, switched on by the vendor as an enhancement, in a renewal cycle where nobody was asked. The March 2026 ARL and CNI workshop report identifies governance of vendor-embedded AI features as the least developed area in research library practice, despite being the most immediate operational reality for most libraries. Its response is a strategy: treat vendor AI features as a governed surface rather than a default benefit.

Stony Brook University Libraries states the problem plainly in its own published AI statement. It acknowledges that many of the tools the library subscribes to now include AI features that arrived without a procurement decision. That is the most useful sentence any library has published on this subject, because it names a condition most libraries share and few have said out loud. The vendor evaluation framework earlier in this module applies cleanly to a product the library is considering buying. On its own, it does not apply to a feature that appeared inside a platform the library licensed three years ago for entirely different reasons.

The practical response has three parts, and the first is an inventory. To govern these features, the library must first know which exist. That means going through the subscribed databases and discovery layers one at a time and recording, for each, what AI functionality is present. Note whether it is on by default, whether it can be turned off, and whether the library was notified before it appeared. For example, a mid-sized academic library doing this exercise will typically find AI summarization, natural language query translation, or an embedded research assistant in several platforms it never evaluated for those capabilities. The inventory is unglamorous and takes a few hours spread across a term. It turns an unknown surface into a known one.

The second part is a preview and testing requirement. A feature that is on by default in a patron-facing discovery layer is, functionally, a service the library is offering. It deserves the testing any new service would get before launch. To test it honestly, run the queries the library's actual patrons run, not the ones the vendor demonstration used, and compare what the feature returns against what you would have recommended. ALA's six data-handling questions from earlier in this module also apply to an embedded feature exactly as they apply to a purchased one. That the library did not choose the feature does not reduce its responsibility for what the feature does with patron queries.

The third part is a set of opt-out criteria decided in advance. Before the library is under pressure to decide case by case, it should state what would cause it to disable a vendor AI feature. Examples include functionality that misleads patrons about the scope of what was actually searched, privacy telemetry that sends patron queries somewhere the library cannot account for, or demonstrated bias in what the feature surfaces and suppresses. The library should also establish whether disabling is technically possible for each feature. The honest answer divides into two layers that behave very differently.

At the visible assistant layer, opting out is real, and it is documented in the vendors' own administrative guidance. Ex Libris states plainly of the Primo and Summon Research Assistant that "by default, this functionality is disabled." An institution running it has therefore switched it on at some point and can switch it off again. JSTOR lets an administrator enable or disable the interactive research tool for the entire institution through JSTOR Admin, and lets an individual user turn it off for themselves. Scopus AI is available only as part of the institutional subscription and is controlled by the institution's Elsevier administrator. The EBSCO API search scope in Primo VE is an optional slot that a discovery administrator can toggle at the institution level. For example, a library that has decided an AI research assistant in the discovery layer does not serve its patrons well can act on that decision itself, today, without waiting for the vendor to offer a choice. One caveat is worth carrying into that work. When the Primo Research Assistant launched, an independent analysis found that hiding the interface link did not fully disable the underlying endpoint; a knowledgeable authenticated user could still reach it. Confirming that a feature is off means testing that it is off, not observing that the link is gone.

The second layer has no switch. AI embedded behind the scenes in relevance ranking, automated tagging, and summarization applied during indexing generally cannot be disabled separately. Vendors describe it as part of how the platform works, not as a feature the library adopted. The inventory should treat this functionality as non-optional rather than as a decision the library deferred. That changes what the inventory is for. The question becomes what the library must disclose to patrons about how results are ordered, not whether it can turn the ordering off.

This work has a collective dimension that individual evaluation does not reach, and the clearest recent case is specific enough to cite in a renewal conversation. In a guest column published on October 24, 2025, a Colorado State University librarian set out the terms CAS, a division of the American Chemical Society, had proposed for SciFinder. One prohibited-use clause would have barred inputting any content accessed through the product "into any Large Language Model (LLM) or related technology regardless of whether such LLM is public or private." That language is broad enough to reach a researcher pasting an abstract into a summarization tool. Advocacy from the CSU community and the national academic library community followed, and CAS proposed revised language. An editor's note added on November 10, 2025, confirms that an agreement resolving the concerns was reached that same month. The Library Copyright Alliance has issued statements and filed formal comments arguing against contractual restrictions that abridge fair use. SPARC has convened libraries specifically to share what has worked in negotiating AI restrictions out of vendor agreements. Montana State University Libraries has published Viewfinder, a values-driven toolkit for deciding whether and how to adopt an AI tool at all. Using any of this does not require leading a national campaign. An ordinary subject or electronic resources librarian can name a specific clause to a vendor representative during a renewal conversation, knowing that peer institutions have successfully objected to similar language. What remains outside the vendor's reach is the library's judgment about whether a feature serves its patrons. That judgment must be exercised whether or not the library was consulted before the feature arrived.