Captured by a human. No algorithms involved.
January is the time for new year’s resolutions. Some surveys suggest ~75% can be attributed to improving our health: exercise, nutrition, and mental health. But the vast majority of us (60-80%) won't make it past mid-February.
While it’s tempting to dive into what this actually means for improving our health, about how digital & AI can facilitate lifestyle changes and increase motivation, let’s flip it around. Roughly 1 in 10 resolutions are about learning something new. And the new thing to learn right now is “AI”. But what does that even mean? And who? And why? And how?
And honestly, while my professional career heavily involves being an AI decision maker, I make no claims about being an ideal AI enthusiast & consumer. So, structuring this and putting it down on paper didn’t seem like a bad idea from my own perspective either!
Let’s start with AI in our professional lives. I think it’s often helpful to go beyond what wonderful things could be done “in theory” and to think through a risk-mitigation lens. Because these days you’re at risk for both using and not using AI!
AI currently can
· Read more papers than you, with better retention.
· Imperfectly structure a much larger amount of knowledge than you.
· Learn how to combine different data types for specific tasks.
· Write & predict faster than you.
· Learn strategies from data, especially when given clear rules and objective.
· Emulate reasoning with increasing complexity!
AI currently cannot
· Reliably reason on par with or in the same way humans can, with accountability.
· Comprehend individual articles as well as a human domain expert: e.g. capturing nuance, context, and emotion.
· While AI can learn strategies, it cannot fully strategize – at least without human guidance and problem definition, boundaries, context, etc. Yet AI is also transforming how strategists work!
So, stemming from what it can and can’t do - how are you at risk for either using it, or not using it?!
Risks of not using AI in life science:
· Incomplete searches or literature reviews within a data source, leading to missing important information. This could be from additional PubMed articles in your field, from a field you’re not familiar with, or from past internal findings, or even a distant work colleague.
· Inability to triangulate between different data types – e.g. connecting ‘omics or pathway information to existing literature and internal knowledge.
· Sub-optimal decision making from your lack of personal background – we can’t all know everything and have had every experience. Many of you can probably identify with this in a drug discovery & development setting, but it holds for physicians as well. Long complicated and incomplete patient histories, the information gap between specialists and GP’s, everything that happens to a patient between visits. Even simply the specialist expertise gathered through years of poring over, say, histopathology images.
· Being too slow. Most notably with real-time alerts from data feeds. Think hospital floors or clinical trial bottlenecks. But also will eventually lead to comparatively poor throughput at work, getting beat to an opportunity by competition, or not processing enough information for a solid decision before a deadline.
· Or, lacking throughput and scale where it matters. (e.g. pharmacovigilance, QC/maintenance, etc)
· Not catching mistakes. We all use spellcheckers. But writing tools can help point out if you missed an important angle in your text. Or if your decision or medical scan interpretation, for example, was not in line with prior cases.
Risks of using AI in life science:
· Learning and/or repeating incorrect information, or information that’s been transformed just enough to lose context for an intended use or logical argument.
· Loss of confidence in others if you sound too GPT-like, or incorrect info starts to pop up making someone doubt the whole argument.
· Fooling yourself – “Brandolini’s law” recounts how much more time it takes to refute misinformation than to create it – we can get unduly biased if we anchor on AI output and assume we can quickly proofread or fact check it before copy/pasting it along, rather than assembling a perspective yourself and using AI to accelerate and fill in gaps.
· Application of tools outside of their scope of generalizability and/or inherent biases in data.
· As an organization, lack of accountability where there is not a strong enough human-in-the-loop
At a high level, that means AI should be thought of as having been given little to no accountability in general situations – models trained for a very specific purpose can be thought of slightly differently – but in almost all cases in health, there needs to be a human-in-the-loop. And we need to understand what that human is there to prevent. Basically, if you were chatting with your very well-read, but eccentric and slightly drunken uncle at dinner, you would need to verify and validate everything they said, but they might expand your line of thinking and give you fresh perspective or ideas. This is AI at its best. That and AI trained to do one thing and one thing only.
In any case, “you” are the only one that can fully de-risk your usage of AI, as it’s becoming increasingly ubiquitous. And I would propose that it helps to understand how the AI was made and what it was optimized to do.
· For example, a current LLM is significantly well-read, but stores everything in an internal registry which may not be up to date, and could even be filled with conflicting information, or gaps it simply fills in, not to mention was not optimized to preserve logic – hence it has very limited transparency and quality controls, but can easily be combined with other AI and other systems. And on the web we do increasingly see references alongside AI text.
· Similarly, in a more professional setting, you might see methods such as RAG, or retrieval-augmented generation – directing the AI via the query to the information it should use to answer a question. Many would say it is much more reliable when AI is leveraged to extract the information from a known source – and can be especially valuable in summarizing very disjoint information such as records.
· On the other hand, some of you might have interacted with knowledge graphs or repositories, which are often used to keep track of pairwise relationships between biomedical entities, are often heavily curated, and produce precise bits of knowledge directed back to precise sources information, albeit the context in which this information holds can still be a challenge. But in this way the AI you’re interacting with – if any – is often just used to write queries and to summarize the results.
· Or you can go a step further, and work directly with raw data – with LLM-style AI again acting as a query/coding and summary interface. In effect, this now democratizes data which your organization might have paid for already, and enables you to specify your question in the most relevant way and context to you, rather than searching for a “similar enough” paper elsewhere or proposing a project to a data science team. All of which you can still do as a cross-check.
I’ve also found it interesting to hear people recount and evaluate how well AI could - or couldn’t - reproduce and expand upon their past work!
A few important points, from the FT, to peer-reviewed journals, to my LinkedIn friends:
· Agentic based approaches mean AI can do more than write and/or have become more specialized at doing so: gathering data, data engineering and coding.
· Validation of output will vary, and can be time consuming itself.
· As discussed, this will further democratize both data insights and scientific ideas.
· This will also facilitate the community in ensuring any given result is reproducible – although the same tech will mean there are many more results to reproduce!
· Given that, the bottom line on signal-to-noise in research publications is yet to be seen.
· There are many publications showing AI failing to do robust autonomous scientific research, but potentially adding value if thought of as, say, research assistants.
· AI is evolving so quickly, there is a gap between the frontier and legacy models, with those at the front having significantly improved handling of task complexity.
· Our value as professionals will be increasingly on the quality of ideas in the near future!
In closing:
· Don't think of AI as a technical skill, but as shift towards a new way of working.
· Understand the strengths and weaknesses, just like you would a new coworker, to make it more intuitive to assess the risks of using / not-using in a given context.
· Pick a first workflow or decision to augment. Where do you feel you could benefit from faster or broader access to data? This is the fun part anyway, and there is a probably an example out there of someone doing so.
· Be clear on your guardrails and your “human-in-the-loop" critical role.
· Work in a way to increase the impact of both your ideas and your critical lens!