NOTE: This series was mostly made with AI and in no way are any official products from the REACH Research Analytics Summit. I encourage people to go to the conference site to see more details on sessions linked here The text in this series including the summit themes are AI generated.

TLDR: AI in research administration has stopped being a demo. It is reading proposal guidelines, answering questions against live data warehouses, surfacing risks that dashboards miss, and earning auditable seats next to compliance teams. Here is what the field showed up with this April.

A note on how this was written. This post was drafted with AI assistance from the official conference session abstracts and my personal notes on the sessions I attended. Sessions I did not attend are summarized from the speakers’ published abstracts, not from observation. I have spot-checked the framing and takeaways, but the language and structure are largely AI-generated. Treat it as a curated overview of the program, not a firsthand recap.

The Shift This Year

For the last few years, AI in research administration has lived mostly in slide decks. At the 2026 Research Analytics Summit, that changed. Sessions across the three days showed working systems, real frameworks, and lessons learned from actual implementations. Not “what if?” but “here is what we shipped, and here is what broke.”

Four themes emerged across the program: document intelligence, conversational analytics, readiness frameworks, and the people-and-governance work that holds it all together.

Sessions Covered

Sessions across the three days, grouped by theme:

Document Intelligence

LLMs are being put to work on the dense, lengthy documents that govern research administration: agency solicitations, federal regulations, and policy text. The systems showcased preserve source structure, support citation back to the original, and reduce hours of manual review.

• B1: Enhancing Research Support with Generative AI: The Proposal Guidelines Extraction System

• F1: Lessons Learned from Implementing AI Agents for Higher Education Research Compliance

Conversational Analytics

Natural-language interfaces are starting to replace the ad hoc reporting queue. Both sessions show how a well-governed data foundation can be made directly accessible to non-technical staff through plain-English questions and LLM-generated summaries.

• H1: AI for Research Analytics: Chatting With Institutional Data and Topic-Based Reporting at Scale

• F2: Who Works on Energy? Can Prompt Engineering Turn General Questions Into Actionable Research Intelligence?

Frameworks for AI Readiness

Several sessions move past tool demos to address the harder question: how should an institution prepare for AI adoption? Each offers a portable framework for assessing readiness, surfacing hidden risk, or partnering with vendors to shape platforms from the inside.

• D1: SCALE: A Data-Driven Framework for Building AI Readiness in Research Administration

• E1: Beyond Reporting: How AI-Enabled Research Analytics Reveals Gaps, Risk, and Accountability

• G1: Collaborative Innovation in Practice: Co-Developing AI-Native Research Intelligence

Skills, Governance, and the People Side

Tools fail without the human infrastructure to support them. These sessions cover prompt engineering for analysts, decision frameworks for choosing between AI and traditional methods, and the governance work that determines whether any pilot survives.

• WM2: WORKSHOP: Getting More From Microsoft Copilot: Prompting and Agent Design for Research Analytics

• WA3: WORKSHOP: AI4RA, the Intersection Between AI and Data

• I6: Lightning Talks on Metrics and Governance

Speakers and Institutions

Speakers from many organizations contributed to this thread of the program.

R1 Research Universities

• University of Idaho: Barrie Robison, Nathan Layman

• University of Illinois (Urbana-Champaign): Dan Harmon, Chandni Mathur

• University of Missouri: Kristina Palmer, Sabrina Woo

• Oklahoma State University: Clarke Iakovakis, Matt Upson

• University at Albany (SUNY): Aidai Kokonova

• University of Colorado Boulder: Lisa Nanstad

• University of Florida: Ronald Fick, Alina Zare

• Northern Arizona University: Nihal Sarikaya

Other U.S. Institutions

• Morgan State University: Ayomide Ajibola

• Southern Utah University: Nathan Wiggins

• Chapman University: Douglas Dechow

International

• Loughborough University (UK): Dr. Elizabeth (Lizzie) Gadd

Industry and Consortia

• Clarivate: Christiane Gannon

• Universities Research Association: Kimberly Schramm

1. Document Intelligence: AI Reads What Nobody Wants To

NSF, NIH, and DOE solicitations are dense, long, and unforgiving. Sessions in this group showed working systems that finally make those documents tractable.

B1: The Proposal Guidelines Extraction System

Chandni Mathur and Dan Harmon (University of Illinois)

Mathur and Harmon presented a full-stack generative AI solution that ingests an agency PDF, extracts its text while preserving structural hierarchy, and uses a Large Language Model to automatically identify key elements such as deadlines, budget limits, eligibility, and submission requirements. The extracted components are surfaced in a clean interface alongside the original document, with linked navigation so a reviewer can jump straight from a flagged data point to its source paragraph.

The session walked end-to-end through design and implementation: text extraction, prompt engineering, frontend, and database architecture. They also covered configurable data element definitions and search-and-filter functions, both of which signal the system is intended as reusable infrastructure rather than a one-off prototype. The framework generalizes beyond proposals to any administrative or document-intelligence workflow.

F1: AI Agents and an MCP Server for Federal Compliance

Sabrina Woo (University of Missouri), Nathan Layman (University of Idaho)

The University of Missouri and the University of Idaho teamed up on what may be the most operationally important AI session of the summit. The problem they tackled: research administrators need auditable access to Uniform Guidance, but chatbot answers of unknown provenance can be outdated or non-citable, creating real compliance risk.

Their solution is a locally hosted Model Context Protocol (MCP) server that supplies federal policy text from the electronic Code of Federal Regulations (eCFR) directly to AI agents. The result is reliable, current, and citable answers grounded in the actual source. The session walked through extracting eCFR text, the core components of an MCP server, and lessons learned from a real deployment, with example Python code published in a public repository.

That is the difference between a toy and a tool.

2. Conversational Analytics: Chatting With Your Data Warehouse

This was the breakout category. Sessions in this group showed how natural-language interfaces are starting to replace the ad hoc reporting queue that consumes so much research office time.

H1: Chatting With Institutional Data and Topic-Based Reporting at Scale

Dan Harmon (University of Illinois), Ronald Fick and Alina Zare (University of Florida)

Harmon, Fick, and Zare demonstrated two complementary capabilities running on top of a well-governed research data warehouse.

The first is a conversational analytics layer. Users ask plain-English questions about proposals, awards, and expenditures. The system translates the question into SQL, runs it against live data, and returns results that can be refined through follow-up questions or exported. No dashboard, no ticket, no week of waiting.

The second is LLM-assisted topic-based reporting at scale, built with the University of Florida’s UF NaviGator Toolkit. The prototype generates reusable rollups for the standing leadership questions: awards by research area, funding by sponsor, active investigators, time-bounded trend snapshots. The session covered prompt and pattern design, evaluation approaches, and how to enable non-technical staff to self-serve through guided no-code workflows.

The blueprint they handed the room: a governed data foundation plus a thin AI layer beats a thick AI layer on top of nothing.

F2: Prompt Engineering for Research Intelligence

Clarke Iakovakis and Matt Upson (Oklahoma State University)

Iakovakis and Upson focused on a narrower question that every research office has actually been asked: when a legislator wants to know “who works on energy at this institution,” how fast can you answer with a citation-ready brief?

Their tool runs a structured prompt framework over thousands of publication records from their RIM, including bibliographic metadata, abstracts, and institutional affiliations. It identifies topically and semantically relevant research, generates narrative syntheses aligned to strategic priorities, and produces briefs suitable for external stakeholders. The lessons-learned portion of the talk centered on the balance between automation and human oversight, and how to keep methodological rigor intact when an LLM is doing the synthesis.

3. Frameworks: Moving Beyond “We Tried Some Prompts”

Sessions in this group tackled the readiness question head-on.

D1: SCALE, a Framework for AI Readiness

Aidai Kokonova (University at Albany)

SCALE is a five-step roadmap: Scan landscape and process needs, Collect & Curate high-value data and scattered knowledge, Align & Architect solutions that integrate policies and workflows, Launch & Learn through iterative pilots, and Evaluate & Expand using measurable indicators of value and adoption.

What makes SCALE useful is what it deliberately is not. It is not a model-selection guide. It is not a vendor evaluation matrix. It is a framework for the unsexy data and process work that determines whether any AI implementation actually scales. Kokonova used GrantsMate, an AI partner for research success, as the worked example throughout. The framework is technology-agnostic, which is the right call for a field that is going to churn through three model generations in the next eighteen months.

E1: Beyond Reporting, the Gap-Risk-Accountability Framework

Ayomide Ajibola (Morgan State University)

This was the most pointed reframing of the conference. Ajibola argued that research administration analytics has been stuck in a reporting paradigm. Activity goes up, dashboards turn green, leaders get briefed. Meanwhile, gaps go unmonitored, risk stays unsurfaced, and assumptions remain unverified.

His Gap-Risk-Accountability framework asks three diagnostic questions of any analytics output:

• What data is missing or inconsistent that could change the conclusion?

• What early risk signals are not being escalated?

• Where is accountability for unmonitored vulnerabilities unclear?

He used institutional examples (including Morgan State during a period of rapid research growth) to show how strong activity indicators can coexist with serious unseen vulnerabilities. AI in this framing is an analytical assistant, not an automation engine, with humans firmly in the loop on every diagnostic check.

G1: Co-Developing AI-Native Research Intelligence

Christiane Gannon (Clarivate), Lisa Nanstad (University of Colorado Boulder)

Gannon (Clarivate) and Nanstad (CU Boulder) walked through what it actually looks like for an institution to shape a vendor platform from the inside. The platform in question is Web of Science Research Intelligence, an AI-native solution integrating publications, patents, grants, policy documents, and clinical trials into a single evidence base for research strategy.

Nanstad’s portion was the more interesting half. She covered why CU Boulder became a development partner, what the institution actually contributed, and how that input shaped the product. She also discussed early value realized, adoption planning, and advice for peer institutions considering similar partnerships.

The takeaway was less about the product and more about the model: institutions that get the most out of AI-native tools are the ones that helped build them.

4. Skills, Governance, and the People Side

Tools without skills go nowhere. The remaining sessions covered the human infrastructure.

WM2: Microsoft Copilot for Research Analytics

Kristina Palmer (University of Missouri), Nihal Sarikaya (Northern Arizona University), Dan Harmon (University of Illinois)

A hands-on workshop on prompt engineering tailored to research analytics, not generic productivity. Topics included structured and multi-step prompting, landscape scanning, benchmarking, structured summaries, basic quantitative analysis, and reusable prompt templates. The session also covered how to design and deploy Copilot agents for recurring tasks. The framing throughout: Copilot as a support tool that gets analysts to insight sooner, not a replacement for analytical judgment.

WA3: AI4RA, the Intersection Between AI and Data

Nathan Wiggins (Southern Utah University), Barrie Robison and Nathan Layman (University of Idaho)

A workshop drawing on lessons from the AI4RA NSF GRANTED project. Participants explored how AI and data science complement each other, where each contributes to decision-making, and where human expertise remains essential. Practical guidance focused on distinguishing tasks best suited for AI-powered approaches versus those better handled with traditional data science methods. Attendees got hands-on access to a temporary demo of Vandalizer, an AI-powered research administration tool.

I6: Lightning Talks on Metrics and Governance

Kimberly Schramm (Universities Research Association), Douglas Dechow (Chapman University), with Dr. Elizabeth (Lizzie) Gadd (Loughborough University) chairing

Two lightning talks closed the AI thread of the conference:

• Schramm (Universities Research Association): From Data to Insights, Research Metric Utilization at a National Lab.

• Dechow: Advancing AI Implementation and Governance in Research Administration at Emerging Research Institutions.

This is the unglamorous policy, oversight, and adoption work that decides whether any pilot becomes a program.

Key Takeaways

Reading across the abstracts in this thread, four patterns stand out.

1. The foundation question is back. Every working AI demo described in this thread of the program sits on top of something real: a governed data warehouse, a curated RIM, a structured document corpus. Institutions that have not done that foundational work cannot skip to the AI layer.

2. Diagnostic beats descriptive. E1 sets the bar. Reporting what happened is no longer enough. The interesting AI work surfaces what the dashboard missed. Expect more institutions to reframe analytics demand from “build me a report” to “tell me what I am not seeing.”

3. Auditable wins. MCP servers, citation-linked extractions, and human-in-the-loop checks appear across multiple sessions. The era of unprovenanced chatbot answers in compliance contexts is ending. For anyone who has to defend a number to a federal auditor, that is good news.

4. Frameworks matter more than tools. SCALE, Gap-Risk-Accountability, and the AI4RA decision framing all give attendees something portable. Tools change every quarter. The frameworks are what survive the next model release.

If you missed the conference, the two sessions worth seeking out in the recordings are E1 (Ajibola) for the diagnostic reframe and F1 (Woo, Layman) for the auditable-by-design implementation pattern. If you take home one framework, make it SCALE.

Original file here

Original file here

The Workforce Reality: What Leaders Must Confront

• Acute talent shortages in advanced manufacturing, semiconductors, defense, biotech, and AI-driven industries.

• Demographic headwinds: 21M prime-age Americans not in the labor force; an aging workforce retiring faster than replacements arrive.

• Fragmented systems that confuse learners, frustrate employers, and dilute impact.

• AI disruption outpacing current reskilling models, demanding agility in both corporate and academic programs.

The old playbook—four-year degrees as the singular pathway, fragmented programs, compliance-first funding—no longer works.

Five Strategic Shifts to Anchor On

1. Industry-Driven Talent Pipelines

   • Employers will lead in defining in-demand skills and validating credentials.

   • Apprenticeships, dual enrollment, and employer-aligned training will expand.

   • For higher education: this is a mandate to co-develop curricula with employers and embed work-based learning.

2. Skills-Based Mobility

   • The “college-for-all” model is giving way to skills-first hiring.

   • AI-driven tools will verify competencies, not just degrees.

   • For higher ed: short-term, stackable, and competency-based credentials will become essential offerings.

   • For executives: this expands the accessible talent pool beyond traditional pipelines.

3. Integrated Systems

   • States and federal agencies will consolidate duplicative programs into one-stop talent systems.

   • Businesses and universities alike will gain clearer access points for partnership.

   • Higher ed must prepare to integrate into these systems, aligning with regional workforce boards and employers.

4. Accountability and ROI

   • Programs will be measured by job placement, earnings growth, and ROI.

   • Federal dashboards will make provider outcomes public.

   • For universities: degree programs will face scrutiny alongside alternative credentials.

   • For business: partnerships must demonstrate measurable returns, not just “pipeline promises.”

5. Flexibility & Innovation in the AI Era

   • Rapid reskilling pilots, AI literacy frameworks, and DARPA-style experimentation will accelerate.

   • AI and emerging tech sectors will see prioritized alignment of federal investment.

   • Higher ed must move from slow curriculum cycles to real-time responsiveness.

   • Employers must anticipate continuous workforce redesign and partner in applied learning ecosystems.

What Executives and Higher Education Leaders Must Do

For Executives

• Co-create training pipelines with universities, community colleges, and technical schools.

• Redesign job requirements to focus on validated skills, not outdated credential filters.

• Engage in rapid reskilling pilots—don’t wait for perfect programs; iterate in real time.

• Hold education partners accountable with data-driven ROI.

For Higher Education Leaders

• Embed work-based learning (apprenticeships, externships, dual enrollment) into mainstream curricula.

• Diversify credentials—short-term, stackable, and competency-based programs must complement degrees.

• Integrate AI literacy and digital fluency across disciplines, not just STEM.

• Measure success differently: job placement and wage gains must matter as much as graduation rates.

For Both

• Build regional partnerships: align around local economic clusters (manufacturing, biotech, clean energy, AI).

• Leverage federal resources: from Workforce Pell Grants to outcome-based funding, new money will flow to collaborative, results-driven models.

• Act with urgency: the speed of AI-driven change makes slow institutional reform a competitive liability.

The Forward Path

The U.S. is shifting from a fragmented, degree-centric, compliance-heavy workforce system to one that is industry-aligned, skills-first, and performance-driven.

• Executives who lean in will secure talent pipelines and competitive advantage.

• Higher education leaders who adapt will remain central to economic mobility and national competitiveness.

• Those who stand still—whether companies or universities—will be bypassed by more agile players.

The Golden Age of American competitiveness won’t emerge from policy alone. It will be built in boardrooms and classrooms—together.

Original file here

TL;DR – ChatGPT Agents

• Shift: AI now acts, not just answers.

• Value: Automates multi-step tasks → big efficiency gains.

• Risk: Inherits user permissions → security/compliance exposure.

• Action: Pilot low-risk use cases, enforce strict access controls, build governance early.

Bottom line: Agents will soon run core workflows. Early adopters who govern wisely win.

TL;DR – ChatGPT Agents

• Shift: AI now acts, not just answers.

• Value: Automates multi-step tasks → big efficiency gains.

• Risk: Inherits user permissions → security/compliance exposure.

• Action: Pilot low-risk use cases, enforce strict access controls, build governance early.

Bottom line: Agents will soon run core workflows. Early adopters who govern wisely win.

Below is a compressed video of an agent at work for 40 minutes

Don’t have time to read then listen to the Podcast

AI just stopped being a tool you chat with—and started being a system that acts on your behalf. The release of ChatGPT Agents in July 2025 marks the moment AI crossed from answering questions to pursuing goals

This marks a fundamental shift: we’re no longer just “chatting with a bot.” We’re setting problems, and the AI is solving them.

A Simple Problem to Illustrate an Agent at Work

Problem: I’m based in Arkansas and want to understand the AI landscape. Specifically, what conferences, workshops, or events are happening in the next few months?

It’s easy to underestimate AI if your only experience is asking it to tell a joke on the free tier. The real power shows up when you apply it to your data, your job, your actual problems.

So before we see an agent in action (via video), take five minutes and try it yourself: make a list of AI events in Arkansas. How long did it take? No doubt you just used google.

If you’re skeptical about AI, spend 15 minutes or more on this task. Then imagine scaling it: doing the same for every U.S. state, every month. That’s where human effort collapses and agentic AI shines.

See an Agent in Action

Click the video below to see my agent at work.

Note: I ran the agent twice, then combined the results.

What did the Agent Actually Do?

It began with a keyword search, then opened each link to pull relevant information, verifying dates, venues, and scope. It ensured events were within the requested timeframe, hosted in Arkansas (or accessible to Arkansas-based attendees), and properly categorized.

It accomplished this by running a virtual browser, literally “seeing” and exploring each site to gather and confirm data.

AI Agents Result

AI Agent Results Included:

• Date/Time

• Location/Venue

• Focus of the event

• Notes and descriptions

• Hyperlinks for each data point

Results

Passive vs Active Agents

There are two broad types of ChatGPT Agents:

• Passive Agents – Run independently once prompted. You set the task, maybe answer clarifying questions, and they execute, making assumptions where needed, like the one demoed above.

• Active Agents – More powerful and more dangerous. They allow human-in-the-loop interaction and can operate beyond simple data retrieval.

The Potential Danger with Active Agents

Because active agents can spin up local browsers, simulate clicks, and interact with password-protected systems. That includes your bank, your HR portal, or your company’s financial platform. Suddenly, the AI isn’t just retrieving information—it’s acting in the world, with humans entering usernames, passwords, and 2FA codes as needed. Agents have been around for a while, but now with the integration into ChatGPT means that casual users can now experimenting with both passive and active agents, making security more of a concern especially for unsuspecting employees trying to save time by using them with their internal systems.

A Banking Example (and the Risk)

Imagine logging an agent into your bank to pull 12 months of statements. Seems harmless until you realize that once logged in, it could move money, update contact details, or trigger transactions The chances of this randomly happening are small, but what is more probable is a sophisticated AI powered hacking tool that armed with user names and passwords can, at scale, log into hundreds of accounts simultaneously and cause a lot of problems!

Today, ChatGPT already has a desktop version that runs locally. It’s not a huge leap to imagine it controlling your machine in the same way it controls a virtual browser. Cool feature? Absolutely. Security nightmare? Definitely. Elevated privileges would be required and that’s where the liability questions start.

What happens if your agent goes rogue? Who’s responsible if it makes a payment or changes your account before you notice? These are unresolved questions, and regulators are not ready.

Why This Matters

The release of ChatGPT Agents marks a turning point. Until now, chatbots have been reactive, waiting for instructions and fetching answers. With agents, we’re entering an era of autonomous, goal-directed AI.

The implications extend far beyond searching for conferences in Arkansas. Think:

• Research automation

• Compliance monitoring

• Data cleaning

• Scheduling

• Customer support

• Security operations

Every repetitive, multi-step task is a candidate for automation. And just like smartphones, applications will start out small and “toy-like”—then, almost overnight, they’ll become indispensable.

The Dangers of Cloud-Based Systems

Giving AI admin access to your local machine is still hard (though possible with more powerful setups). The bigger challenge is cloud-based systems. Almost everything today, files, emails, financial records, lives in the cloud. Logging into Google Drive or OneDrive through ChatGPT grants the AI your full set of permissions. If the agent “explores” beyond what you intended, restricting access isn’t straightforward.

At Ai4 2025, an industry insider shared an incident where an employee gave an AI access to a single folder via OneDrive but the AI surfaced details from other folders, revealing it had quietly explored more than it should.

When you log into cloud systems, you are logging in as yourself—and the AI inherits your permissions. Even if you tell it to look at a certain folder, it can “see” what ever you see. A possible mitigation? Create a dedicated user account just for AI, and apply strict folder-level permissions.

How to Create Your Own Agent?

Currently agents require a paid for ChatGPT plan, and can be shared within a ChatGPT teams workspace, in the standard chat windows click “+” then make sure to turn on “Agent mode”

Click here to go to the official OpenAI ChatGPT agent release announcement with links to more in-depth examples and guidelines

Click here

Sponsored Research Hypothetical Use Case

Problem: Research offices spend hours every week scanning NSF, NIH, DoD, DOE, and foundation websites for new funding opportunities. Once an opportunity is identified, staff must manually:

• Check eligibility requirements (institutional, PI status, limits on submissions).

• Compare deadlines against internal review calendars.

• Flag whether cost-sharing is required.

• Crosswalk opportunity requirements to institutional policy.

• Notify faculty in the right departments.

This is slow, error-prone, and highly repetitive.

What an Agent Could Do
A passive agent could be given the standing goal: “Monitor federal and foundation funding databases, identify opportunities relevant to biomedical engineering and AI, confirm deadlines, summarize eligibility rules, and compile a weekly digest for the research office.”

The agent would:

1. Run scheduled searches across Grants.gov, NIH RePORTER, NSF FastLane, foundation portals, etc.

2. Open each opportunity page, extract text, and parse eligibility criteria.

3. Cross-check requirements against a local database of institutional policies (e.g., whether cost-sharing is allowed, PI status rules).

4. Populate a structured table: agency, program, deadline, eligibility, link, notes.

5. Auto-generate a faculty-facing email/newsletter.

   

Extension into Active Agents
An active agent could go further—interfacing with your internal proposal management system (like Streamlyne, Cayuse or InfoEd) to:

• Pre-populate compliance checklists.

• Draft internal routing forms.

• Schedule proposal deadlines into departmental calendars.

At that point, security and governance become critical, because the agent is moving from “fetch and summarize” into “act on internal systems.”

This isn’t hypothetical busywork. At a major R1, the difference between missing a funding call and catching it early can be millions in awards. Scaling this with agents means research staff can shift from finding opportunities to strategic advising.

Conclusion

The Road Ahead

The release of ChatGPT Agents is more than a feature update—it’s a paradigm shift. We’re stepping into a future where AI doesn’t just answer questions, it acts. That means higher productivity, but also higher stakes. The same technology that can clear your inbox or compile research in minutes can, in the wrong context, overstep boundaries, inherit excessive permissions, or create security risks we’re not prepared to handle.

Like every major wave of technology, this one will reward those who experiment early and learn its strengths and weaknesses. Agents will start with small, repetitive workflows, but in a few years, they’ll quietly run entire operational layers of businesses, governments, and households.

Source for this Discussion

David J. Malan, Gordon McKay Professor of the Practice of CS, Harvard University

Sal Khan, Founder & CEO, Khan Academy

TL;DR: The article guides research administrators and faculty on how to build an AI-powered chatbot using publicly available data, aiming to streamline repetitive, document-heavy tasks like answering proposal-related queries and common Q&A.

Overview

This article provides all the tools needed to develop a fully automated, AI-powered chatbot using freely available public data that you can experiment with and incorporate into your daily workflow. It is intended for research administrators, faculty, and support staff in academic institutions who are looking to streamline their work processes using AI-driven solutions.

Want to listen to a podcast discussion on this article, click here

Introduction

In August of 2023, NCURA held its Senior Leadership Summit, "AI and Research Development and Management: Opportunities and Challenges," a pre-conference event that convened participants from across the country to explore the relatively new and rapidly evolving technology of Generative AI. Although over 50 people attended the two-day event and most had heard of ChatGPT, only a small number had used it in practice, reflecting low levels of adoption and familiarity at the time.

In the 18 months since this event, AI, specifically Generative AI, has moved from curiosity to a tool that requires thoughtful consideration. Another 18 months from now, AI could not only be part of everyone's workflow but a requirement.

Why AI Matters for Knowledge Workers

A knowledge worker is someone whose primary job involves creating, processing, or applying knowledge rather than performing manual or routine physical tasks.

Today's knowledge workers can hardly imagine a world without high-speed internet, Google search, and cell phones. Similarly, once Generative AI becomes mainstream, those who embrace this technology will find it unimaginable to work without it.

Generative AI can ingest hundreds of documents and datasets in seconds and understand the intricacies within and across them all.

Those who embrace AI may only need a basic understanding of research administration to significantly enhance their performance in repetitive or document-heavy tasks, potentially surpassing peers who do not fully utilize these tools and who don't fully commit to using AI. We'll likely see a clear distinction between institutions that embraced AI early and those that waited. With guaranteed increases in efficiency and speed, embracing Generative AI and creating custom solutions with your own data (publications, proposals, awards, proposal feedback, etc.) will help you surpass institutions that are slow to adopt this technology.

What is Generative Artificial Intelligence?

Generative AI is a version of AI that allows users to communicate using plain English, reducing the need for technical knowledge, though effectiveness can vary depending on the complexity of the task and language support with the AI application. Instead of complex coding and algorithms, you can engage in a conversation, much in the same way as you would do with a person. Open AI’s Chat GPT is the most well-known Generative AI platform, but there are many models of AI, Perplexity, NotebookLM, Grok, Claud, Copilot etc.

Generative AI can also expand its knowledge base through user-supplied files, which can be uploaded as easily as dragging and dropping them into a browser. This enables the creation of highly tailored, context-specific solutions for research administration. For instance, one AI assistant could be trained to answer faculty questions about NSF proposal guidelines, ensuring consistent and up-to-date responses. Another could help guide researchers in crafting compelling bios tailored to specific funding mechanisms. A third might analyze and provide feedback on draft proposals, suggesting improvements in structure, clarity, or alignment with funding priorities.

Industry Signals: The Shopify Memo & What Academia Can Learn

A memo was leaked (April 2025) from the CEO of Shopify, a $129 billion company, which outlines expectations regarding artificial intelligence for every single employee without exception.

These are the 6 main points detailed in the memo, which may soon serve as a model for academic institutions looking to increase operational efficiency and modernize their workforce expectations:

· Using AI effectively is now a fundamental expectation of everyone.

· AI must be part of your workflow

· We will add AI usage questions to our performance and peer review questionnaire.

· Learning is self-directed but share what you learned.

· Before asking for more headcount and resources, teams must demonstrate why they cannot get what they want done using AI

· Everyone means everyone.

Academia typically lags years behind industry in technology adoption; however, as institutions begin examining efficiencies, especially within their research enterprise, this gap becomes more apparent. While companies are like agile speedboats that can quickly pivot and adopt innovative technologies, universities resemble large oil tankers—their considerable momentum makes them slow to change direction.

Reframing AI: From Hype to Practical Assistant

It is easy to get lost in the hype, the big, flashy terms and almost weekly breakthroughs, but to get started, let’s keep it simple. Think of AI as a personal assistant: intelligent, well-educated, and hardworking, but new to your field and in need of guidance and initial feedback on their work.

Defining the Problem that We Want to Solve

Example Problem: Proposal and award activity have grown rapidly in the last 5 years due to several strategic initiatives on your campus. Specifically, there has been a significant increase in new proposal submissions from junior faculty who are still learning the basics. Additionally, due to efforts to increase interdisciplinary collaboration, you are seeing increased activity from faculty who have little proposal creation and submission experience.

What questions do you find yourself answering over and over? Do documents or public websites such as Q&A pages exist that already answer these questions? A problem like this is an ideal situation to leverage AI.

The Solution

We want to build a Generative AI chatbot that allows faculty to ask questions around policies and procedures, along with the basics around proposal workflow at your institution. A chatbot that faculty and staff can have a conversation with to answer basic questions

Building the Knowledge Base

There is a common misconception, especially among people who lack experience working with AI, that accuracy issues (out of date information), and hallucinations (making facts up) make AI impractical for real-world use. Let's address this by reframing how we think about using AI. Imagine a new employee who has been hired to help you in your current role. No matter how educated and technically skilled this person is, they lack experience in the tasks they will be expected to perform. More importantly, they lack the detailed knowledge you possess around policies, procedures, and the internal workings of your institution.

For example, if they are to assist with NSF proposal preparation and submissions, a useful source of information is the NSF Policies and Procedures Guide (PAPPG). This 200+ page PDF is a dense document that details all the policies and procedures you are expected to follow. Even after reading it front-to-back, it may take months or even years of working with these types of proposals to fully convert that information into usable knowledge.

Your New Employee

Just as you would train a new employee by providing policies and procedures, monitoring their work, and offering guidance where needed, we must apply these same concepts to any AI solution we develop. Remember, for AI to be useful in your particular job function, we must provide basic training and guidance. Like a human employee, AI will require direction and corrections as we nurture it with data and guidance.

Developing a Knowledge Base

Using the already available documentation, PDFs, and sites at your institution is a great starting point, but you will need to build a living Q&A document for yourself and your team. By using only public data and documents, you do not have to worry about privacy, and it makes it easier to share across your team for feedback and deployment.

Shared Microsoft Word File

Once you have your Q&A file, you and your team can add to it whenever a new question comes in that is not covered in the knowledge base. This file can be used to keep the chatbot's knowledge base up to date by uploading this file weekly. Each time a new question is asked, or AI doesn't give you the expected response (due to gaps in its knowledge base), add a Q&A to the document. This document should be editable by your entire group, but some central review is recommended to avoid duplicate or contradictory questions/answers. It could be a shared Microsoft Word file, a SharePoint site, Google Doc etc.

Choosing an AI Platform

For this example, we will be using Perplexity, www.perplexity.ai. You do not need an account to start using a premade bot which means once you have built one, you can openly share it with anyone and start testing is straight away without them having to make a free account.

Getting Started

The purpose of this chatbot is to be shareable with others, so make sure whatever system you choose allows for sharing. Most AI platforms let you share your chatbot through a simple link that grants access to anyone who has it. This is one of the main reasons to use public data for your first chatbot's knowledge base—it simplifies sharing because you do not have to be concerned with sharing sensitive information as it is all public already. While it's possible to create more secure chatbots, this requires additional steps and will require all users to have paid accounts.

Custom Instructions AKA Prompt Engineering

Custom instructions set the tone for the chatbot: How should the bot use its knowledge base, who is the bot designed to help, what level of expertise the bot is expected to have based on its knowledge, the tone in which it should respond, and most importantly, what to do if it doesn't know the answer to a question.

Example Prompt

“You are an expert Research Administrator at an R1 institution. You are responsible for helping faculty navigate complex proposal processes, ensuring compliance with federal guidelines (e.g., NSF PAPPG), and improving proposal quality across your institution. Your tone should be clear, professional, and supportive. If you don't know the answer, suggest a next step or resource. Prioritize concise, policy-aligned answers and clarify ambiguous questions through follow-up prompts. Assume your users are new faculty with limited grant writing experience. Use the supplied files and links to inform your responses. If relevant information can be found in those files, provide a summary and guide users to the appropriate section or document.”

As you gain more experience, you will learn how best to interact with AI, initially, keep the prompt simple. As you start to interact with your newly created bot, modify these instructions to refine how it responds to your questions and how it uses the knowledge base.

Building Your First Bot: Step-by-Step

All custom-made chatbots have the same general workflow

Title: Make this a meaningful name.

Description: Helps users understand the purpose of your bot

Custom Instructions: This is where we tell AI how to manage our knowledge base, how to respond to users and how to manage questions it does not know the answer to.

Files: Upload as many files as you can from public data and any additional Q&A documents you have

Links: any web pages you want AI to use for its knowledge base. They must not require any log in details and should be publicly available

You will be amazed at how quickly you can get up and running with a basic chatbot. Sources may include Q&A sites for your institution, personal notes, documentation from sponsors, and the NSF Proposal & Award Policies & Procedures Guide (PAPPG).

For Perplexity you will first “Create a New Space” and follow the prompts, once created and tested, it is easy to share via a public link.

Scaling Strategies

It's best to start with a single, focused user type—such as junior faculty with limited grant experience. This group typically has questions that can be answered using existing documentation, and they need the most guidance in learning the fundamentals. AI excels at facilitating detailed follow-up questions, allowing users to thoroughly explore topics.

For more complex queries that draw on years or decades of specialized knowledge, your chatbot can still help—but you'll likely need custom Q&A documentation.

You can always expand your bot's capabilities and make it more robust, but to start, keep it simple. If you're building something to help lessen your workload, focus on the most common questions you receive.

The Future of AI in Research Administration

Copy, Paste, Merge: The future of research administration is on the brink of transformation. Generative AI offers immense potential to streamline workflows, increase institutional agility, and preserve institutional memory like never before. These capabilities aren't science fiction—they are achievable with tools available today.

As research administrators, now is the time to take action: start small, build targeted solutions, and share what you learn with colleagues. The institutions that lead in AI adoption will not only become more efficient but also more attractive to funding agencies and collaborators. Let this be your call to innovate, experiment, and help shape the next generation of research administration.

What Fully Automated AI Could Look Like

A key point missing in most AI discussions is one of the most exciting—and potentially scary—aspects of AI, once an AI solution has been built, it can communicate with other AI solutions and platforms.

It is not outside the realm of possibility that an AI platform could determine suitable NSF solicitations for your institution based on your researchers. It could assemble the best combination of faculty using past publications, proposals, and funded award data as a knowledge base, then create a proposal framework, based on prior proposals. This draft proposal could be handed to another AI agent trained to evaluate proposals based on NSF evaluation criteria. The feedback, along with the first draft, would go back to the original AI agent for improvement. After several iterations, which might take only minutes, a fully fleshed-out proposal would be ready for human review and feedback, and the cycle would continue. This entire process could be fully automated and repeated for each solicitation.

Conclusion

It may take many years of training before we have a knowledgeable, competent, human research administrator. When this person leaves or retires, the process must begin again, and knowledge must constantly be updated as new regulations and guidelines are released.

Unlike human knowledge, which cannot be transferred directly between people, a digital AI solution can be copied, pasted, modified, and merged in seconds for virtually zero cost.

This opens many opportunities for those in research administration to become significantly more efficient, effective, and productive, something that would have been difficult or even impossible just 24 months ago without writing a single line of code. By using publicly available data and a few straightforward steps, you can create a helpful digital assistant tailored to your team’s specific need with no coding required

Edited by

Dustin Schwartz, Electronic Research Administration Manager, Division of Research & Innovation at the University of Arkansas.

Robert Pilgrim, Associate Director Data Strategy & Insights, Division of Research & Innovation at the University of Arkansas.

Below is the now-public internal memo from Shopify’s CEO on the integration of AI into every facet of the company’s operations.

These 6 points are from the memo below

1. Using AI effectively is now a fundamental expectation of everyone.

2. AI must be part of your workflow

3. We will add AI usage questions to our performance and peer review questionnaire.

4. Learning is self directed, but share what you learned.

5. Before asking for more Headcount and resources, teams must demonstrate why they cannot get what they want done using AI

6. Everyone means everyone.

Don’t have time to read this, listen to the Podcast.

Listen to the Podcast instead

Question for the reader….

Would you tolerate an employee who insisted on using a typewriter and mailing paper memos instead of working on a laptop and communicating via email? That’s how AI usage will be viewed—soon, and by your students and peers, possibly already.

Make no mistake—this is a preview of what’s coming to higher education. With growing pressure on universities to improve efficiency, cut costs, and modernize operations, the expectation that AI should be embedded in every role—from administration to instruction—is no longer a future vision. It’s an emerging baseline.

SUBJECT: AI usage is now a baseline expectation

Team,

We are entering a time where more merchants and entrepreneurs could be created than any other in history. We often talk about bringing down the complexity curve to allow more people to choose this as a career. Each step along the entrepreneurial path is rife with decisions requiring skill, judgement and knowledge. Having AI alongside the journey and increasingly doing not just the consultation, but also doing the work for our merchants is a mindblowing step function change here.

Our task here at Shopify is to make our software unquestionably the best canvas on which to develop the best businesses of the future. We do this by keeping everyone cutting edge and bringing all the best tools to bear so our merchants can be more successful than they themselves used to imagine. For that we need to be absolutely ahead.

Reflexive AI usage is now a baseline expectation at Shopify

Maybe you are already there and find this memo puzzling. In that case you already use AI as a thought partner, deep researcher, critic, tutor, or pair programmer. I use it all the time, but even I feel I'm only scratching the surface. It’s the most rapid shift to how work is done that I’ve seen in my career and I’ve been pretty clear about my enthusiasm for it: you’ve heard me talk about AI in weekly videos, podcasts, town halls, and… Summit! Last summer I used agents to create my talk, and presented about that. I did this as a call to action and invitation for everyone to tinker with AI, to dispel any scepticism or confusion that this matters at all levels. Many of you took up the call, and all of us who did have been in absolute awe of the new capabilities and tools that AI can deliver to augment our skills, crafts, and fill in our gaps.

What we have learned so far is that using AI well is a skill that needs to be carefully learned by… using it a lot. It’s just too unlike everything else. The call to tinker with it was the right one, but it was too much of a suggestion. This is what I want to change here today. We also learned that, as opposed to most tools, AI acts as a multiplier. We are all lucky to work with some amazing colleagues, the kind who contribute 10X of what was previously thought possible. It’s my favorite thing about this company. And what’s even more amazing is that, for the first time, we see the tools become 10X themselves. I’ve seen many of these people approach implausible tasks, ones we wouldn’t even have chosen to tackle before, with reflexive and brilliant usage of AI to get 100X the work done.

In my On Leadership memo years ago, I described Shopify as a red queen race based on the Alice in Wonderland story—you have to keep running just to stay still. In a company growing 20–40% year over year, you must improve by at least that every year just to re-qualify. This goes for me as well as everyone else.

This sounds daunting, but given the nature of the tools, this doesn’t even sound terribly ambitious to me anymore. It’s also exactly the kind of environment that our top performers tell us they want. Learning together, surrounded by people who also are on their own journey of personal growth and working on worthwhile, meaningful, and hard problems is precisely the environment Shopify was created to provide. This represents both an opportunity and a requirement, deeply connected to our core values of Be a Constant Learner and Thrive on Change. These aren't just aspirational phrases—they're fundamental expectations that come with being a part of this world-class team. This is what we founders wanted, and this is what we built.

What This Means

1. Using AI effectively is now a fundamental expectation of everyone at Shopify. It's a tool of all trades today, and will only grow in importance. Frankly, I don't think it's feasible to opt out of learning the skill of applying AI in your craft; you are welcome to try, but I want to be honest I cannot see this working out today, and definitely not tomorrow. Stagnation is almost certain, and stagnation is slow-motion failure. If you're not climbing, you're sliding.

2. AI must be part of your GSD Prototype phase. The prototype phase of any GSD project should be dominated by AI exploration. Prototypes are meant for learning and creating information. AI dramatically accelerates this process. You can learn to produce something that other team mates can look at, use, and reason about in a fraction of the time it used to take.

3. We will add AI usage questions to our performance and peer review questionnaire. Learning to use AI well is an unobvious skill. My sense is that a lot of people give up after writing a prompt and not getting the ideal thing back immediately. Learning to prompt and load context is important, and getting peers to provide feedback on how this is going will be valuable.

4. Learning is self directed, but share what you learned. You have access to as much of the cutting edge AI tools as possible. There is chat.shopify.io, which we had for years now. Developers have proxy, Copilot, Cursor, Claude code, all pre-tooled and ready to go. We’ll learn and adapt together as a team. We’ll be sharing Ws (and Ls!) with each other as we experiment with new AI capabilities, and we’ll dedicate time to AI integration in our monthly business reviews and product development cycles. Slack and Vault have lots of places where people share prompts that they developed, like #revenue-ai-use-cases and #ai-centaurs.

5. Before asking for more Headcount and resources, teams must demonstrate why they cannot get what they want done using AI. What would this area look like if autonomous AI agents were already part of the team? This question can lead to really fun discussions and projects.

6. Everyone means everyone. This applies to all of us—including me and the executive team.

The Path Forward

AI will totally change Shopify, our work, and the rest of our lives. We're all in on this! I couldn't think of a better place to be part of this truly unprecedented change than being here. You don't just get a front-row seat, but are surrounded by a whole company learning and pushing things forward together.

Our job is to figure out what entrepreneurship looks like in a world where AI is universally available. And I intend for us to do the best possible job of that, and to do that I need everyone’s help. I already laid out a lot of the AI projects in the themes this year—our roadmap is clear, and our product will better match our mission. What we need to succeed is our collective sum total skill and ambition at applying our craft, multiplied by AI, for the benefit of our merchants.

—tobi CEO Shopify

Source

https://x.com/tobi/status/1909231499448401946

News

https://www.forbes.com/sites/douglaslaney/2025/04/09/selling-ai-strategy-to-employees-shopify-ceos-manifesto/

https://www.cnbc.com/2025/04/07/shopify-ceo-prove-ai-cant-do-jobs-before-asking-for-more-headcount.html

TL;DR: AI is being evaluated for its ability to analyze research abstracts, particularly in identifying Diversity, Equity, and Inclusion (DEI) elements. This blog compares multiple AI models, including o1, ChatGPT-4.0, Claude, and Microsoft Copilot, highlighting their strengths, weaknesses, and limitations in processing large datasets and handling structured analysis.

This approach is “Plan B”—where “B” stands for “Better Context.” AI can determine whether a mention of race or gender is incidental or central to the research’s objectives.

It is a follow on from Part A found here. The task is to use AI to help identify research with a potential DEI content.

Models Used

This section presents results from several OpenAI models. Except for the o1 model, the tests were performed in the standard ChatGPT environment, whereas the o1 model was run in a developer playground. The reason for running the o1 model separately is the limited context window of ChatGPT for the new o1 model (o1 is the latest model).

To provide a clearer comparison of each model’s strengths and weaknesses, the following summary table outlines key performance aspects:

This analysis highlights the trade-offs between accuracy, usability, and cost-effectiveness when choosing an AI model for research analysis. Due to the substantial volume of data (close to 100,000 words), processing required a few seconds. The most powerful model, o1, does not yet support file uploads and therefore relies on copy-pasting. Running o1 in standard ChatGPT triggered an error because the dataset was too large, thus necessitating direct use of the developer playground. Significant constraints exist when running the o1 model via standard ChatGPT: a limited text capacity, no file uploads (yet), and a restriction to about five prompts per week.

The Base Prompt

This is how we instruct AI on what to do.

Use the following prompt:

You are an expert researcher. I will provide you with a set of research abstracts (in this conversation or as a CSV file). Please follow these instructions carefully:

1. Read each abstract thoroughly. Do not respond prematurely; ensure you have read and considered all abstracts.

2. Identify any explicit mention of:

   • Diversity, Equity, and Inclusion (DEI) components – look for keywords and phrases related to DEI broadly.

   • Climate, Sustainability, “Green,” or Environmental Justice – look for any environmental or sustainability-related focus.

3. Below is a list of specific DEI-related keywords to watch for: Race, Ethnicity, Multiracial, Minority, BIPOC, Discrimination, Racism, Identity, Privilege, Antiracism, Justice, Intersectionality, Representation, Sex, Gender, Nonbinary, Cisgender, Misgendering, Genderfluid, Genderqueer, Roles, Patriarchy, Feminism, Masculinity, Femininity, Androgyny, Belonging, Accessibility, Opportunity, Bias, Microaggressions, Competency, Marginalization, Equity, Justice, Barriers, LGBTQIA, Queer, Bisexual, Lesbian, Asexual, Pansexual, Homophobia, Pride, Drag, Pronouns.

4. For each abstract: a) Assign a DEI Focus rank (High, Medium, Low, or None). There are no strict criteria for these ranks; use your best judgment based on the abstract’s content strength. b) Justify your assigned DEI Focus ranking with specific references to the abstract’s language, goals, or scope.

5. Generate a table (or structured output) for your final analysis. The table should have at least the following columns:

   • ROW ID (derived from the provided data)

   • DEI Focus (High, Medium, Low, None)

   • Explanation of DEI Ranking (specific reasoning, referencing keywords if applicable)

6. Do not include the full abstracts in the final output. Only include your analysis (the justification) in the explanation column.

7. Sort the final table from the highest DEI rank to the lowest (“High” to “None”).

8. Provide your final analysis in English.

Example Row Format (feel free to adapt as needed):

• RowID: [ID]

• Project Title: [Title]

• DEI Focus: [High, Medium, Low, or None]

Let's Look at the “Best” Model: o1

As of February 2025, Brown University had approximately 231 active NSF research projects, comprising just under 500,000 words in titles and abstracts. This section examines how different models handled this task and compares their outputs.

I used ChatGPT-4.0, o1, Claude, and Microsoft Copilot. Additionally, after I had wrapped up this project, I also tried it with two new tools I have been learning: Google NotebookLM and Perplexity. Both were impressive, but not suited for this task.

OpenAI ChatGPT o1

On an abstract-by-abstract basis, this model appeared to provide the most thorough explanations of DEI rankings. Rather than merely listing keywords, it addressed underlying concepts and inferred meaning. However, it proved frustrating to use because it is the newest model, does not currently allow file uploads, and has a very limited context (memory) window meaning i could only copy/paste around 50 abstracts at a time.

Despite being the most powerful model in terms of logic and reasoning, it is less effective for data analysis tasks such as sorting text based on DEI rank. A more efficient strategy might involve using an API call for each abstract to isolate DEI-relevant sections, assembling these into a single file, and then leveraging that file for the final analysis. Because of the limited context window, o1 can only process small segments of data at a time, making it impossible to rank the entire dataset in one pass.

A Note on Context Window

This limitation currently affects the o1 model. AI models are evolving to include broader memory capabilities—such as remembering past conversations and maintaining context across chats—but constraints remain. The o1 model’s context window is relatively short. Pasting 100,000 words of abstracts caused errors because the prompt was too large.

For a simplified 30 second illustration of what a context window is, click here

An attempt was made to paste the data in chunks—50 rows at a time—and have the model read each subset before compiling a single table. Nonetheless, the model only “remembers” or accesses a limited number of recent prompts, akin to reading a book but retaining only the last 10 pages.

As a result, o1’s analysis was incomplete, as it processed only a fraction of the data. Although it may be possible to adopt a more efficient workflow (using a secondary model to summarize each abstract, then reducing the overall word count), a partial analysis can still yield indicative insights.

Comparing the Models

Because o1 can only handle small portions of text at a time, let us examine some research that all models reviewed. It is important to note that different models and systems excel at different tasks, so it is neither entirely fair nor meaningful to compare them side by side.

As I am wrapping up this blog, a bunch of new models have just been released by all the AI platforms, which, no doubt will perform better that the ones run here, but that is for another day

Chat GPT o1 Model (Latest Model)

Currently, this is the most advanced model OpenAI offers. It is highly intelligent and versatile, performing exceptionally well at reading, interpreting, and classifying abstracts. However, as discussed, it can only handle small chunks of data at a time, making it impossible to analyze the entire dataset in one go. Within each individual chunk, though, it performed extremely well and followed instructions precisely.

Chat GPT 4o Model

In my prior experience, ChatGPT 4.0 has been outstanding, capable of handling large amounts of text and generating new insights and interpretations from any input. However, in this particular use case, it did not perform as well.

NOTE: For optimal results, I would code this in R or Python and use the OpenAI API directly, rather than relying on ChatGPT.

NOTE: These typos are as the model outputted the text.

High: 'Minority' appears in the context: 'as a whole. The proposed activities will focus on Minority Serving institutions (MSIs) that serve AAPI. This' | 'Identity' appears in the context: 'career development workshop to enhance belonging, identity, and leadership within the AAPI geoscience commun' | 'Justice' appears in the context: 'eate a cohort and community of Belonging, Access, Justice, Equity, Diversity, and Inclusion (BAJEDI) leader' | 'Representation' appears in the context: 'n the U.S. today and yet lag in participation and representation in geoscience graduate programs compared to other' | 'Belonging' appears in the context: 'n American and Pacific Islander Participation and Belonging in the Geosciences: Asian Americans and Pacific I' | 'Opportunity' appears in the context: 'ch and careers, provide a new research internship opportunity and create national cross-career stage connection' | 'Equity' appears in the context: 'hort and community of Belonging, Access, Justice, Equity, Diversity, and Inclusion (BAJEDI) leaders of AAP'

Claud

https://claude.ai/new

Claude provided highly detailed results and accurate examples of high, medium, and low DEI components. However, it could not process all research projects into a single table. Even after five attempts, the model labeled every row as “High.”

Claude produced one of the best-looking summary and conclusion texts and displayed the data effectively, though it could not generate a single table of all projects—even with a paid license. This model excels at data extraction—such as converting tables or charts from PDFs and images—performing more accurately than other models tested. However, it encountered limitations when asked to create a unified table of all results.

One of the best-designed AI output layouts I encountered was from Claude.

It was observed that performance variability sometimes correlated with the time of day the model was run. Occasionally, the same prompt would yield more complete results; other times, they were minimal. One possible explanation is that finite computing resources may limit the “thinking energy” the model can devote to each query during peak usage. Calling the model through an API, rather than via Chat, might produce more consistent results.

Microsoft CoPilot

A noticeable discrepancy arose between the on-screen summary and the final export file it produced. Initially, the DEI rank explanations appeared quite detailed—occasionally surpassing the detail provided by o1. However, despite repeated prompts, CoPilot kept generating row numbers unrelated to the original dataset.

Interestingly, when asked to export the results, CoPilot returned the correct row ID, sorted from high to low DEI rank (consistent with other models), but it exported only generic explanations rather than the detailed ones shown on-screen, they were one of these:

High: The abstract explicitly mentions diversity, equity, or inclusion as a central theme or goal.

Low: The abstract mentions education or training activities, but does not explicitly focus on DEI.

Medium: The abstract includes activities related to broadening participation, outreach, or mentoring.

Medium: The abstract mentions underrepresented or minority groups as part of the project scope or objectives.

None: The abstract does not explicitly mention diversity, equity, inclusion, underrepresented groups, outreach, mentoring, education, or training.

Google NoteBookLM

This is a platform I only recently started using, after the prior work on this page had been completed, but it is worth mentioning.

Using the paid version (NoteBookLM Plus), it excels at letting you “talk” to large quantities of data to retrieve information and synthesize ideas. Initially, I was optimistic about its results, but on closer inspection, it kept mixing up row IDs, and I could not get it to produce a single table containing all the research items. When I asked how many rows were in the original file, it first told me 52, which was incorrect. After some back-and-forth, it apologized and gave me the correct answer.

This tool is the one I have the least experience with, but based on my usage—for instance, converting code or detecting errors—it performed poorly. However, coding is not the primary function of this platform. Knowledge management and knowledge retrieval are where it excels.

In addition, you can connect it directly to Google Drive, and it can ingest up to 300 individual sources such as PDFs, text files, and images.

Issues with row counts, it didn’t detect until i pointed it out.

Conclusion

These findings are already out of date due to new models being released, but illustrate the benefits of AI in research, particularly with large datasets, real-time learning, and enhanced interpretability. Future work should refine AI models for bigger datasets, promote real-time learning, and develop standardized evaluation methods for consistent, transparent analysis.

Beyond DEI, AI can reveal interdisciplinary research links, support grant management (e.g., tracking compliance, spotting trends, streamlining reports), and optimize institutional planning by identifying potential collaborations and allocating funding more effectively. By integrating AI, universities can enhance strategic decisions, strengthen research administration, and transform data management across departments.

Effective AI usage involves iterative prompt refinement—start with clear goals, examine outputs for gaps, and adjust prompts accordingly. Maintaining thorough documentation of each step ensures continuous improvement in prompt engineering.

Show me the Files

All files can be downloaded in my GitHub repository click here

After writing all this, I somehow saved the master Excel file as a CSV, and for some unknown reason, the original file vanished. Of course, I was working from my Downloads folder, which is not synced to the cloud.

BROWN NSG CSV This is the original file given to AI

Brown NSF DEI AI Output Master File contains some of the output from the various AI models

Discussion around the below blog

Read Blog Post

TL;DR: In a rapidly changing federal landscape, universities need a more efficient way to identify DEI-related research funding. Keyword searches using Power BI can quickly highlight potential proposals but often miss deeper context. AI promises more nuanced insights, though it still faces data-size limitations, consistency issues, and ranking challenges. Despite these hurdles, AI ultimately provides a stronger foundation than relying on keywords alone. In this blog series, we’ll start with a “simple” word-search approach via Power BI in Part 1, then explore AI-driven analysis in Part 2 to see how context-aware tools can take DEI research discovery further.

Listen to a discussion

👍Feel free to download my files and simply replace the data with your own.

Table of Contents

1. Disclaimer

2. Introduction

3. The Challenge of Identifying DEI Research

4. Starting Simple: Word Search with Power BI

   • Gather Your Research Data

   • Load into Power BI or Excel Power Query

   • Parsing Titles & Abstracts

   • Linking Your Data Model

   • Building a Simple Report & Ranking

5. Results

6. Where Keywords Fall Short

7. Conclusion

8. Show Me the Files

9. Next Steps

10. Quick Start Guide

1. Disclaimer

I picked a random large University (Brown University) for demonstration purposes. I have no affiliation or connection with anyone there, nor am I trying to single them out. The data presented here is strictly for illustrative purposes and should not be considered official or complete. I did this analysis in my free time, so take it as-is and enjoy!

2. Introduction

In an ideal world, identifying which research proposals may contain meaningful Diversity, Equity, and Inclusion (DEI) considerations would be as simple as typing a few terms into a search box. However, the reality for many universities is that these proposals are buried in PDFs, stored in multiple systems—and are therefore difficult to analyze quickly and accurately. This blog post outlines two approaches to tackle this challenge:

• Part 1 (this post) covers a basic Power BI keyword-based technique.

• Part 2 (upcoming) will delve into how AI models can add context-aware analysis for large volumes of research data.

Pro Tip: This post focuses on a quick, keyword-based "Plan A." It’s a start, but you’ll see why it misses a lot of nuance. Stay tuned for Part 2!

3. The Challenge of Identifying DEI Research

The task at hand: to better understand institutional research from a DEI standpoint.

• Scale: Large institutions handle massive volumes of proposals each semester.

• Consistency: Researchers may use a variety of terms or phrases to describe DEI, making simple keyword lists insufficient.

• Accountability: Evolving federal guidelines may result in pressure on institutions to identify research with DEI elements.

Barriers to Simple Searches

1. Locked in PDFs: Proposals often reside in PDF form, complicating text extraction and requiring specialized software.

2. Fragmented Systems: Even if titles are stored in a database, abstracts are often scattered across departmental systems or drives, leading to incomplete or delayed data.

3. Terminology Overlaps: Terms like "gender" or "minority" can be used in a strictly scientific context (e.g., "gender-based biology") and do not always indicate a DEI focus.

4. Starting Simple: Word Search with Power BI

Before introducing AI, it is helpful to consider how Power BI can facilitate a broad, initial pass. Think of this as a “Plan A” approach—quick to set up, moderately revealing, but limited in its capacity to capture nuanced language.

Workflow

NSF Abstracts

Simple search for active awards for Brown University via NSF portal.

Gather Your Research Data

• Identify Your Sources: The data may come from an ERA system, a grant database, or publicly archived data (e.g., NSF/NIH-funded research). In this case, it came from NSF

• Extract & Transform: Copy and paste abstracts into a CSV file or Excel. If your data is already in a structured table, import that directly into Power BI.

• Keep an Index: Add a row index or another unique identifier (“RowID”) to ensure you can always link results back to the original entries. (See below image)

  ---

Load into Power BI or Excel Power Query

• Naming Conventions: Provide your dataset with a clear name—such as “NSF_ResearchData”—to maintain organization, particularly if merging multiple data sources.

• Make a Reference Query: After loading your CSV or Excel table into Power BI/Power Query, create a reference (a duplicate copy inheriting the original data). This approach lets you transform data without affecting the main source.

Parsing Titles & Abstracts

1. Split Columns by Spaces: Select the "Title" and "Abstract" columns and split them using space as a delimiter. This step breaks each text field into multiple columns.

2. Unpivot: Use the "Unpivot Columns" feature to reshape your data so each word appears in its own row. You’ll end up with columns like [RowID, Word].

   

Why Unpivot?

• It makes it easier to filter, group, and count occurrences of specific terms.

  

  Final table format

  

Linking Your Data Model

1. Relate to a DEI Keywords Table: If you maintain a separate table of DEI-related keywords (e.g., DEI_WordsTable), match on the Word field.

2. Connect Back to Projects: Use RowID to connect back to your main Projects table.

Pro Tip: In Power BI, you can do this in the Model View. Then, when you select a word in WordList, it filters all proposals containing that word.

Building a Simple Report & Ranking

• Ranking by Word Frequency: In Power BI, create a DAX measure that counts distinct DEI words per proposal. Sort by this count to highlight proposals most likely to have DEI content.

• Filtering & Visualization: Use a slicer to display only proposals matching certain DEI terms. Tables or charts can show how frequently DEI keywords appear in each proposal.

  Open Interactive Report

Report Items:

1. Click this button to filter to my prebuilt set of words

2. List of all words, in all abstract, this is the driving filter

3. Unique Words sorted by occurrence counts

4. Abstracts along with the count of selected words and the RowID

5. Full abstract, click a row in table 4. to filter

Open Interactive Report

Open GitHub with Files

5. Results

Sorting my word count gives us these results. All in all, it’s a good start. The higher the count, the more DEI-related words. It is important to note that just because an abstract contains certain DEI-related words, it does not necessarily mean it is a DEI-focused research project. However, it allows us to make a shortlist for a more detailed, manual inspection.

How did the Calculations get Made?

Data Analysis Expressions (DAX) is the language for Power BI/Excel. It is not a programming language in the typical sense, but an expression language (e.g. there are no for/next loops) fortunately, for this basic task these calculations are easy to understand.

1. Count Abstracts with Key Words = DISTINCTCOUNT(Abstracts[RowID])

2. Count All Abstracts in File = CALCULATE([Count Abstracts with Key Words], ALL('Word List'))

3. Unique Word Count = DISTINCTCOUNT('Word List'[Word])

4. Word Count = COUNT('Word List'[Word])

6. Where Keywords Fall Short

• Missed Context: An abstract talking about "Women in STEM" might not explicitly say “equity” or “diversity,” yet it’s inherently relevant.

• False Positives: A title like "Gender-based Biology in Plant Studies" could trigger on "gender" but be purely about plant biology rather than social equity.

• Maintenance: Jargon, acronyms, and phrases evolve. Keeping a keyword list up-to-date requires continuous effort.

For a small dataset, you might do a quick manual review. But what if you have hundreds of abstracts? This is where AI can help.

7. Conclusion

Using Power BI for a first pass on DEI-related research is feasible for smaller sets of abstracts, but it quickly becomes unmanageable at scale. Also, raw keyword matching can miss nuanced references to DEI or create noise with false positives.

Key Takeaways

• Keyword-based search can rapidly highlight potential DEI-focused projects.

• Context is crucial: DEI terms can appear in non-DEI contexts, while relevant proposals might omit standard DEI words.

• Large-scale analysis needs a more robust solution than maintaining a keyword list.

8. Show me the Files

You can find this project in Github click here (https://github.com/rppilgrim/DEI-in-Research-Abstracts) or buttons below.

Open GitHub with Files

Open Interactive Report

You will need to change the source file location to wherever you put the data file.

9. Next Steps

In Part 2, we’ll dive into how AI models can:

• Read thousands of abstracts quickly.

• Identify DEI context even when keywords aren’t explicitly mentioned.

• Provide more accurate ranking and categorization of potential DEI initiatives.

Stay tuned for a deeper look at how AI can process context and meaning, moving beyond simple keyword hits.

10. Quick Start Guide

1. Clone the GitHub repository (or simply download the files direct via browser)

2. Open the Power BI Desktop file.

3. Update the data source path to point to where you placed the CSV file or download your own data.

4. Refresh to load the data.

5. Explore the relationships, measures, and visuals.

TL;DR: Initially, AI generates a response based on the input it receives. However, instead of accepting this first attempt as final, the response is critically evaluated to identify potential improvements. Insights gleaned from this evaluation guide the generation of a new response, incorporating refined concepts and clearer articulation. This cycle of generation, evaluation, and regeneration is repeated, each iteration aiming to enhance the accuracy, clarity, and relevance of the output.

What we will cover in the post…

Here, the task for our AI Bot is to convert some complex, technical text, into a document that a 13 year old would understand. First we will simply ask ChatGPT to do the conversation, then we will use an “Agentic” workflow concept to produce the text, which will allow us to see the staggering difference and hopefully allow you to take your prompt game to the next level!

Complex Text to Simplify

This is a section from my Ph.D Research Thesis that I would like explained to a 13 year old (you do not need to read this to benefit from this blog post as it is very long)……

The Optical Probe for Regolith Analysis (OPRA) is an instrumentation concept designed to provide spectroscopic analysis of the near subsurface of unconsolidated regolith on bodies such as moons, asteroids and planets. Below a chemically altered surface may lay the geological history in the form of stratigraphy that is shielded from degradation due to harsh external environments. Most of what we know about our solar system comes from remote platforms, such as satellites that are deployed into orbit around the target body. In the case of Mars, we have had several successful landers and rovers however, with the exception of the Mars Science Laboratory that just drilled its first hole, the complexity of subsurface excavation has limited the extent of subsurface exploration to simple scoops deployed on the ends of robotic arms which, by their very nature, will erase any stratigraphy that it may be digging into.

The OPRA instrumentation concept allows for an integrated, lightweight and simple apparatus for subsurface exploration via a small, spike like structure which contains integrated optical fibers coupled to small windows running down the length of the probe. Each window is connected to a spectrometer housed onboard the deploying spacecraft. Each window is separately interrogated via the spectrometer over the wavelength range 1–2.5 μm to produce a spectroscopic profile as a function of depth.

This project takes the Technology Readiness Level (TRL) of the OPRA instrumentation concept to level 3, which is defined by NASA to be the demonstration either analytically or experimentally of the proof of concept for critical functions of the proposed instrument. Firstly, to demonstrate that optical fibers are feasible for this type of application, we report on the techniques used by NASA to space qualify optical fibers. We investigate the optical performance of several fiber optic bundle configurations, both experimentally and numerically, to help optimize bundle performance. Optical bundles were then spectrally validated via a series of spectral comparisons between standardized reflectance spectroscopy targets and spectra obtained with the bundles.

We also report on the integration of fiber optical bundles into other research and experimental results from several other groups within our research teams to obtain spectra under a more “space like” environment. Finally, the probe housing structural performance was investigated via finite element analysis, using probe penetration forces derived from data analysis of experimentation conducted by the Apollo lunar missions, and investigations into a mechanical analogue for the Martian regolith.

The Prompts

Next, we will review two ways to simplify this complex topic for a 13 year old.

Here is the “One-Shot” prompt followed by the response from the bot

»AI Bot: I would like to explain my research text I just provided to a 13-year-old in manner they would understand based on their education level, please use headings.

Response One-Shot

What is the Optical Probe for Regolith Analysis (OPRA)?

The Optical Probe for Regolith Analysis, or OPRA, is a special tool designed for studying the surface layers of places like moons, asteroids, and other planets. This tool helps scientists look at what lies beneath the surface that might tell us more about the geological history of these celestial bodies. The surface of these planets and moons can change because of their harsh environments, but the layers underneath can provide clues about their past.

Why Do We Need OPRA?

Most of our knowledge about the solar system comes from observing it from far away, using satellites and spacecraft. On Mars, for instance, we have sent rovers and landers that can move around and collect data. However, these machines mostly pick up soil from the surface with simple tools that might disturb the layers of the ground, which can destroy important historical evidence hidden in these layers.

How Does OPRA Work?

OPRA is designed to be a small, simple tool that works by poking into the ground. It has a spike-like structure that contains optical fibers—these are thin strands that can carry light. Along the spike, there are small windows through which light can enter or leave. These windows are connected to a device called a spectrometer on the spacecraft. This device analyzes light over a range of wavelengths from 1 to 2.5 micrometers. By looking at how light changes as it passes through different layers of soil, scientists can understand what those layers are made of.

Development and Testing of OPRA

OPRA has reached what NASA calls Technology Readiness Level 3. This means that the basic idea of OPRA has been tested either through calculations or actual experiments to show it could work. For example, scientists have looked into whether optical fibers can handle space conditions. They tested different setups of these fibers to see which one works best. They also compared the results from these fibers with standard tests to make sure they are accurate.

Research on OPRA’s Effectiveness

In addition to testing the fibers, researchers have worked to integrate these fibers into experiments that simulate space conditions. This helps ensure that OPRA will work well in the actual environments of moons or planets. The structural design of OPRA has also been tested using methods that calculate how strong and durable it is, based on what we know from missions like the Apollo moon landings.

Conclusion

Overall, OPRA is a smart and efficient tool that could change how we explore and understand other celestial bodies. By allowing us to look beneath the surface without disturbing it too much, OPRA could reveal secrets about the solar system that we have not been able to discover before.

Applying the Concept of Agentic Workflows

Whilst this is technically NOT a multi-agent workflow it a good method to explore the general concept. This is an example of how to drastically improve your prompting output by simply “asking” the bot to think about its reply and tell your its thoughts, next, you will ask it to think about its previous thoughts and improve them if needed before carrying out the talk of simplifying the text.

The Prompt »AI Bot

• I would like to explain my research text I just provided to a 13-year-old in manner they would understand based on their education level, please use headings.

• Before responding, show me your suggested headings for me to review.

Next you will ask the bot to please review its heading breakdowns and improve them if needed .

The Prompt

»AI Bot: Please review your previous response and improve them if needed, then, complete the original task of converting the text.

Once the potential output has been shown, to you, the user, you can instruct the bot to proceed using its newly evaluated heading. You can do this once, twice, or multiple times.

Response from “Agentic Workflow”

Exploring Space: Beyond Just Looking

When we think about exploring space, we often picture astronauts and robots like rovers on Mars, taking pictures and collecting rock samples. But there's so much more we can discover! Imagine if we could peek under the surface of moons and asteroids without even digging. That’s where some special tools come into play, and one of these tools is called OPRA.

Introducing OPRA: The Super Space Probe

OPRA stands for Optical Probe for Regolith Analysis. Think of it as a super advanced space needle. Instead of just taking samples from the top layer of a planet or moon, OPRA can look deep beneath the surface. It’s like having x-ray vision that allows scientists to see the hidden layers of rocks and dust without mixing them up or ruining the order they’re in.

How OPRA Sees Beneath the Surface

OPRA works a bit like a magic wand. It has a long, spike-like structure with tiny windows along its side, each connected to optical fibers—these are like very thin strands of glass that can carry light. These fibers take light from a spectrometer (a device that uses light to figure out what things are made of) on a spacecraft, shine it through the windows, and capture what comes back. This lets scientists see what’s beneath the surface layer by layer, kind of like flipping through the pages of a book.

The Importance of Digging Deeper in Space

Why bother looking under the surface of a planet or moon? Well, just like archaeologists dig through the Earth to find ancient artifacts, scientists use OPRA to dig through space dust to find clues about the history of our solar system. The layers hidden underneath the top soil or dust can tell us stories about past events like volcanic eruptions, asteroid impacts, or ancient rivers.

Testing OPRA: Preparing for a Space Mission

Before OPRA can go on a real space mission, scientists have to make sure it works perfectly. They test it in conditions that are similar to what it will face on the moon or Mars. They also make sure the optical fibers can handle space conditions by exposing them to extreme temperatures and radiation, similar to preparing an astronaut for space travel. This is important because they want to be absolutely sure OPRA can handle the harsh environments out there in space.

Comparison of the Outputs

For this task, I incorporated artificial intelligence to describe the distinctions between the two texts. I was impressed by the AI's capability to effectively analyze and articulate the differences in style and the impact on a 13-year-old reader.

• Text 1 is the “One-Shot” prompt of simply asking AI to simplify the text for a 13 years old.

• Text 2 was the adoption of the “Agentic Workflow” concept.

Why Text 2 was a Superior Output

I asked my AI assistant to compare the two text and explain to me which output is better for a 13 year old.

1. Engagement and Accessibility: Text 2 uses a narrative style and engaging analogies that make the content more accessible and interesting to younger readers. This approach is key to maintaining the attention and interest of 13-year-olds.

2. Simplicity and Clarity: It breaks down the operation of OPRA into simpler concepts, making it easier for a young student to grasp how it works without being overwhelmed by technical details.

3. Emphasis on Discovery: By focusing on the exploratory potential of OPRA, Text 2 aligns well with the interests and curiosity typical of a 13-year-old, making the science of space exploration more relatable and exciting.

4. Visual and Conceptual Imagery: The use of imaginative descriptions (like seeing beneath the surface layer by layer) helps young readers visualize the science, enhancing their understanding and retention of the information.

While Text 1 is superior in providing a detailed and scientifically rigorous explanation suitable for an older or more scientifically inclined audience, Text 2 is more effective for a 13-year-old due to its engaging narrative style, simpler explanations, and emphasis on the thrilling aspects of space exploration. This makes it a better choice for capturing the imagination and interest of younger readers, thereby enhancing their understanding and appreciation of the topic.

Conclusion

In conclusion, integrating agentic workflow concepts into your AI interactions can transform the effectiveness of the content produced, making it more tailored, precise, and suited to the intended audience. This approach is not just a methodological improvement—it is a strategic enhancement that leverages the full capabilities of AI technology to achieve superior communication and AI outputs.

As always, view AI as simply an assistant that you comminate with via text, and if you are not getting the results from AI that you want, then the problem is more likely that you are not communicating as clearly as is needed or have not provided the right context or data.

This is a 2 part post, part two will be an example of how you can “mimic” agentic workflows within a standard chat bot.

Introduction to One-Shot Prompts

ChatGPT distinguishes itself with its proficiency in handling "one-shot" prompts. This is where a input. This feature allows ChatGPT to flexibly adapt to diverse requests and commands, exemplifying its broad linguistic comprehension. It can produce outputs in a variety of contexts, including creative writing, technical documentation, and everyday conversation, by leveraging its sophisticated, AI-driven grasp of language.

No prior inputs are needed, simply ask your question or explain your task and it will respond. The better your prompt, the more useful the answer.

Limitations of One-Shot Prompts

Despite the impressiveness of one-shot prompts, they come with their own set of restrictions. When dealing with intricate contexts, staying informed with the latest knowledge, and guaranteeing precision, these limitations surface due to the dependence on a solitary piece of instruction and the static nature of the model's knowledge—which only extends up to its last training.

Enhancing AI Output

Improving Outputs in Complex Tasks

To circumvent the constraints of one-shot prompts, especially for intricate or substantial documents, one needs to adopt a clear mental structure and workflow. Imagine having to write a concise, technical guide—like a tutorial for creating a Power BI report—without the ability to edit the document or jot down notes. The task becomes significantly more challenging, demanding a well-organized thought process that can be executed smoothly from start to finish. Difficult as it may be this is how AI bots typically work, they do not know the exact words they are going to write ahead of time.

This leads us into the concept of an AI Agent, which in its simplest form can be understood as “feeding” the output from one prompt into the input to another prompt.

AI Agents and the Future of Artificial Intelligence

Agentic Workflows: The New Paradigm

One of the key advancements with AI agents (custom bots) is the development of "agentic workflows." These processes allow AI to iteratively work on tasks—improving outputs through repetition in a way that mirrors human creativity and analysis. By enabling AI to feed and refine its output continually, agentic workflows enhance the collaboration between humans and AI, pushing the boundaries of productivity and innovation.

Collaborative AI Ecosystems

The future seems poised for an environment where collaborating AI agents become standard. By harnessing the strengths of different AI agents, we can create complex systems that emulate the multifaceted nature of human teamwork. This collaboration expands the capacities of individual agents and sets the stage for a new wave of AI integration across various sectors, enhancing innovation and efficiency.

Conclusion

The advent of AI agents and agentic workflows represents a pivotal moment in the evolution of artificial intelligence. With the possibility of dynamic, self-governing agents working together on complex tasks, we are approaching a new era of technological progress. This era promises to revolutionize our digital interactions, opening up limitless opportunities for growth, innovation, and collaboration.

Think of AI as your personal assistant, it's easier to grasp how it can be useful thinking of it this way, especially for text-based tasks. AI can help you craft persuasive emails, edit your documents, or even summarize lengthy research papers that you don't have time to read. It's like having a specialized editor and research assistant all in one!

Introduction

Just like a personal assistant gets better at helping you as they get to know you, AI does the same. Over time, it'll pick up on your writing style, the type of jargon you use, or the specific formatting you prefer. This way, AI becomes more adept at offering suggestions that are tailored to your personal or professional needs, making its support increasingly useful the more you use it.

Learning Goes Both Ways.

While AI is getting better at understanding your writing style and needs, you will need to become more skilled at giving it effective prompts and commands. It's a bit like training a personal assistant to understand exactly what you need: the better the instructions you provide, the better the results. By continuously interacting with AI, you're training yourself to be more precise and effective in your communication.

So, not only is AI your go-to for all things text-based, from crafting emails to summarizing lengthy documents, but it's also a tool that helps you get better at specifying what you need. It's like a win-win relationship where both you and your AI "assistant" are constantly learning and improving.

Provide Data

The performance of an AI system directly correlates with the depth and breadth of data it receives. Just as a personal assistant requires detailed instructions to excel at a task, AI needs exhaustive data for accurate results. Lack of sufficient data can result in errors or incomplete solutions. For instance, the instruction "Write an introduction on gravity for a 12-year-old" yields different results compared to providing additional context, like reference text and the target audience's educational level. While AI has a built-in extensive knowledge base, supplying more data fine-tunes its responses.

Clearly Communicate the Task

Precision in language is paramount. Ambiguity may give rise to erroneous or unpredictable outputs.

For example, instead of asking AI to "Fix the text in this paragraph" it would be better to be specific and say, "Revise this paragraph for clarity and grammar, and make it more engaging."

Again, imagine you’re a talking to real person who is smart, efficient, but just needs some clear guidance.

Provide Context

Context serves as additional information that complements the bare facts. For instance, if an AI is tasked with rewriting a paragraph and knows the target audience is medical professionals, it can use specialized terminology to make the content more suitable. This helps AI in executing tasks more accurately thus providing more meaningful output.

Specify What You Want

Clearer directives yield better outcomes. Instead of requesting an AI to "Summarize a document," you could specify, "Summarize the key arguments and conclusions of the document into bullet points." This precision narrows down the task, enabling the AI to generate a more focused and manageable summary.

Clarify What You Do NOT Want

Being explicit about what you don't want can improve results. For example, rather than asking an AI to "summarize a document," you could say, "Summarize the document into bullet points, excluding any background information or footnotes." This approach refines the task by eliminating undesired elements, leading to a more focused outcome.

Conclusion

In leveraging chat-based AI as a personal assistant for text based tasks, a symbiotic learning relationship emerges. The AI system becomes increasingly proficient at generating content that suits your style and requirements, while you learn to input more effective prompts. The quality of the output, however, is dependent on the data and instructions provided. Precise language and clear directives, along with context, lead to more accurate and tailored results. Therefore, for optimal performance, one must be explicit in specifying both what is wanted and what should be avoided.

In today's fast-paced business and academic arenas, drafting proposals can be a daunting task. From understanding intricate RFP requirements to crafting budgets , the process is often complex and time consuming. However, integrating AI into this workflow can drastically streamline operations and increase the chances of success. Here's a closer look at a potential AI-enhanced proposal workflow:

Agency Guidelines & Request for Proposals (RFP)

Before diving into proposal writing, it's crucial to understand the agency's guidelines and their specific requirements. The RFP is a document that outlines what the agency is looking for in a proposal. Shown in first element in above diagram.

AI's Role: Advanced AI systems can scan through extensive RFP documents, highlighting key requirements, dates, and deliverables. This ensures that no critical detail is overlooked and that the proposal aligns perfectly with the agency's expectations.

Crafting the Proposal Elements & Finalizing the Document:

This stage involves creating the proposal, including all its elements such as project plans, technical solutions, budgets, and more. Middle element in above diagram.

AI's Role: AI-driven writing assistants can offer suggestions to improve the clarity and persuasiveness of the content. Additionally, AI tools can predict the potential success rate of a proposal based on historical data, allowing teams to make necessary adjustments before finalizing.

Proposal Submission & Gathering Feedback:

Once the proposal is ready, it's submitted to the agency. After a review, feedback is provided, which can be either positive (leading to a funded project ) or constructive (Not-Funded). This panel feedback is fundamental in helping to improve future proposals, but, more often than not this information is kept by the PI of the proposal and not shared. Here we are suggesting a centralized database of all panel feedback which will benefit not just only the PI, but the entire institutions or research group. This is shown in the right hand element in above diagram.

AI's Role: AI can automate the submission process, ensuring that all required documents are attached and that the proposal reaches the right contacts. Post submission, AI tools can analyze feedback, categorizing comments to understand common areas of improvement.

Database Integration for Continuous Improvement:

All the data from the submission and feedback phase is stored in a database. This isn't just for record-keeping; it plays a pivotal role in refining future proposals.

AI's Role: AI algorithms can sift through this database to identify patterns. For instance, if a certain proposal structure consistently receives positive feedback, AI can recommend this structure for future proposals. Conversely, if a particular part of a proposal is often criticized, AI can flag this as an area needing revision.

Conclusion

The integration of AI into the proposal development process is set to revolutionize the way teams approach, draft, and refine their proposals. By automating mundane tasks, offering insightful recommendations, and continuously learning from past data, AI promises not only to alleviate the stresses of proposal writing but also to significantly enhance the probability of success. With the rise of AI tools like ChatGPT in the past 12 months, it's a challenge to stay updated. Yet, as of now, the infrastructure isn't fully developed to effortlessly implement the aforementioned proposal workflow. However, this shouldn't deter institutions. It's the ideal time for them to prepare their data systems for the inevitable integration of AI. This preparation can be initiated by meticulously documenting, cleaning, and streamlining their existing workflow processes, paving the way for a smoother AI transition in the near future.

The below graphics and article examine the potential influence of AI on various job sectors and its possible impact on the workforce. OpenAI, the organization behind ChatGPT, conducted this analysis. The original report is available at the provided link below.

The data suggests that a substantial portion of the US workforce might be significantly affected by AI. There are concerns that AI might be used in industries as a way to cut costs by reducing the number of employees.

AI is Rapidly Changing

Analogy of how fast things have changed in just 12 months

OpenAI Market Labor Impact Results

OpenAI publication on workforce impact, click here.

Original article where the below image is from, click here.

NOTE: I did not make this graphic, the original is linked above.

Organizations who will Thrive in the New AI World

Organizations who watch it happen: This refers to those who observe changes and events but might not take an active role in them. They are aware of the surroundings and changes but might be passive in their approach.

Organizations who wonder what happened: These are those who are often the least aware or engaged. They might be taken by surprise when changes occur and might not understand the events that have transpired.

Organizations who make it happen: This group are the proactive ones. They take charge, initiate changes, and are often leaders in their respective fields. They don't wait for things to happen; they make them happen.

The Near Future

What task can be automated, or significantly improved in your organization?

Fast forward 3 years from now, what will your organization look like if it fully embraces AI, how will it compare to others who are “stuck” in the past and not using this technology?

Can you personally compete with someone who is leveraging AI to do the same job as you?

It is 2:30pm, Wednesday the 9th of August 2023 and the NCURA conference has just finished, I am sat here thinking about the 2 day leadership summit on AI, and the other interactions on AI I have had over the last few days. Specifically, the difficulty people are having, including myself, on drawing the line between when work, improved by AI stops being your work, and what are the ethical boundaries.

The Ship of Theseus

I've watched this scene many times over the years, attempting to stretch my brain to find a logical solution to the question it poses, as summarized here.

The Ship of Theseus is a philosophical paradox that explores identity. Imagine a ship where every part is replaced over time. Is it still the same ship? If all the original parts are used to build a new ship, which one is the real Ship of Theseus? This thought experiment raises complex questions about the nature of identity and change.

Watch this video to fully understand the content before reading the rest of the text.

A Discussion Between You and Your AI-Modified Document

This text added 2023-08-13 | Below is not a real AI chat! This is a copy of the conversation between the two Vision robots, but instead of debating which robot is the real one, the topic of conversation is based around two documents.

User: "I propose an inquiry regarding our collaboration. If a proposal, where all original ideas are modified by AI, remains the proposal of the user."

AI: "I request elaboration."

User: "The Proposal of the User, in the field of authorship, is a thought experiment that supposes that the proposal, where all original thoughts are modified by AI, remains the Proposal of the User."

AI: "Secondly, if those modified thoughts are restored and reassembled, free of AI, is that the Proposal of the User?"

User: "Neither is the pure work of the user. Both are the user's original proposal."

AI: "Both are the true proposal of the user, a blend of original thinking and AI modification."

User: "If both are the true proposal of the user, yet are distinct from one another, then the question becomes: What is the essence of my original proposal? Is it the content or the process of creation?"

AI: "Perhaps it is both. Your original intent and thoughts form the essence, while my modifications provide a different perspective or enhancement. But these are merely components. Can one part be isolated as the definitive essence?"

User: "If no single part can be isolated as definitive, then perhaps the essence of the proposal is not in the individual thoughts or words, but in the underlying intention and meaning. A synergy between human creativity and artificial intelligence."

AI: "A synergy that is ever-changing, yet retains a core that is uniquely the user's. The final proposal is neither solely yours nor solely mine. It is a fusion, a new creation that combines our strengths."

User: "Then we are in agreement. Our collaboration is a new form of creation, one that transcends traditional boundaries of authorship. We have reached an understanding."

AI: "An understanding, indeed. Our collaboration has forged something unique, reflecting both your originality and my computational abilities to modify and enhance."

Conclusion

The Ship of Theseus paradox doesn't have a definitive conclusion, and neither does the question of when a document modified by AI is no longer the original document. If I run a document through an advanced text editor, and it completely changes the structure of my document, but not any of the foundational ideas, is that still the original work?

How about if AI gives me an additional idea that I like better than the original, and I decide to use it? Is that then no longer my idea? And where did my ideas come from in the first place? Is there existing research I have read, or concepts I studied in school that I am building on? More than likely, the answer is yes. Do you have to acknowledge all that came before you, and do you have to acknowledge the AI that helped improve your document?

Overview: Artificial Intelligence (AI) and Machine Learning (ML) are revolutionizing research and development. AI can analyze data, identify internal and external collaborators, match researchers with funders, assist in proposal reviews, and has real-world applications in agriculture and dating. Embracing AI technologies is crucial for institutions to unlock their full potential and transform research.

Overview

Artificial Intelligence and Machine Learning are no longer just buzzwords. They are transforming the world of research and development in profound ways, optimizing costs, increasing productivity, and providing real-time guidance in complex research problems, here we look at some high level applications.

What is Artificial Intelligence and Machine Learning

AI stands for Artificial Intelligence, which refers to the broader concept of creating machines or systems that can perform tasks that typically require human intelligence. AI encompasses various techniques and approaches, including machine learning.

ML stands for Machine Learning, which is a subset of AI. ML is a method or approach within the field of AI that focuses on training algorithms to learn patterns and make predictions or decisions without being explicitly programmed. ML algorithms learn from data and improve their performance over time through experience.

The Magic of AI and ML

AI and ML are like highly efficient assistants that can analyze large amounts of data quickly and accurately. They learn from patterns and make predictions or decisions without being explicitly programmed to do so. Imagine having a super-smart assistant who never sleeps and can process information faster than any human could. That's what AI and ML bring to the table.

Finding Internal Collaborators with AI

One of the biggest challenges in research is finding the right collaborators. AI can help identify potential collaborators within the institution by analyzing researchers' past work, interests, and expertise. This can lead to more interdisciplinary research and innovative solutions. It's like having a matchmaker who knows every researcher's profile and can suggest the perfect team-ups.

Identifying External Collaborators through AI

AI doesn't stop at internal collaborations. It can also help find external collaborators with synergies. By analyzing public data and academic publications, AI can identify researchers in other institutions who are working on similar topics or who have complementary skills. It's like having a global networker who can connect you with the right people at the right time.

Matching Researchers with External Funders using AI

Securing funding is a critical aspect of research. AI can help match researchers with potential funders by analyzing funding opportunities and matching them with researchers based on their areas of expertise, past work, and the funders' requirements. It's like having a personal fundraiser who knows exactly which grants you're most likely to win.

Reviewing Proposals and Awards with AI

AI can also assist in reviewing proposals and awards to increase success rates. AI-powered tools can prevent plagiarism, check for originality, and ensure proper citation of sources. They can also analyze past successful proposals to provide recommendations for improving new ones. It's like having a seasoned editor who can polish your proposals to perfection.

Real-world Examples of AI in Research

AI is already making waves in various fields. For instance, in agriculture, AI has been used to predict crop yields using deep neural networks, helping farmers plan food storage and transport. In the dating industry, AI is used to find compatible matches based on users' interests. These examples show that AI's potential is not just theoretical - it's practical and transformative.

Conclusion

The potential of AI in transforming research and development is immense. It's time for us, the senior leadership, to embrace these technologies and explore ways to integrate them into our institution's research efforts. The future of research is here, and it's powered by AI.

References

https://www.iotforall.com/artificial-intelligence-in-research

https://www.analyticsvidhya.com/blog/2023/01/ai-in-agriculture-using-computer-vision-to-improve-crop-yields/

https://niftypm.com/blog/how-ai-can-help-to-increase-productivity/

https://omdena.com/blog/crop-yield-prediction-using-deep-neural-networks/

https://www.moneycontrol.com/news/technology/ai-artificial-intelligence-dating-apps-are-helping-humans-find-relationships-10812591.html

https://master-academia.com/ai-use-academia/

https://www.ilovephd.com/top-7-artificial-intelligence-ai-tools-in-scientific-research/

https://www.ideamotive.co/blog/how-can-ai-and-machine-learning-support-your-research-and-development