AI Innovations: The End of Excel Hell in Pharma Supply Planning

Just as Personalized Medicine tailors drugs to individuals, 'Personalized Software' could soon tailor algorithms to the specific needs of individual supply planners.

The needs of planners in pharma differ greatly from other industries. Consider the fundamental question of whether it's better to err on the side of overproduction or underproduction? In many industries it's better to underproduce, for example in aerospace--you shouldn't make too many airplanes. But in pharma, a grandparent who can't get their medication could face life or death. To overcompensate, pharma companies tend to overproduce. But overproduction leads to expiration.

Why is it so hard to balance between stock-out and expiry in the pharma industry?

Planning in the pharma industry is difficult because the off-the-shelf planning software doesn't recognize the planning priorities that are specific to the pharma industry. That software is generic, it's horizontal, it tries to serve all industries. But when you try to appeal to everyone, you end up serving no-one. This software has to be customized to use in the pharma industry and that can take 6-12 months and cost upwards of a million dollars. Many pharma companies resort to using spreadsheets.

It's no wonder that medication stock-outs reached a 10-year high last year, while over $12.5 billion worth of medication expires before being sold each year, on the same order of magnitude as discarding all the toothpaste in the world!

At ABC-Plan, we recognized the need for a supply planning solution tailored specifically for the pharmaceutical industry. Our customers appreciate our software because they can start planning after just one day of training, rather than waiting 6-12 months for implementation. We don't even charge for "implementation"--it's not in our vocabulary. They're saving hundreds of thousands of dollars by avoiding inventory write-offs and stock-outs. And as one customer put it, "using ABC-Plan, I finally have a regular bedtime".

What made it possible for us to focus on pharma is an innovation in funding models. Traditional software companies are horizontal because venture capitalists need 1 in 10 of their investments to be block-buster unicorns. The VC model doesn't allow for vertical focus. But customer-based funding does. By following the "Practical Founder" model, we've maintained our freedom to focus on the pharma vertical.

But what if we could focus even more from the pharma vertical down to the individual needs of specific pharma companies or specific planners on specific days? Not even the Practical Founder model allows for that because until today that type of delivery is custom software development. But that could all change soon.

The next leap forward in personalization could come from AI. Taking inspiration from the "Personalized Medicine" revolution which has been enabled by genomics, perhaps ABC-Plan could harness Artificial Intelligence to provide individual pharma companies with "Personalized Software". Taking it even further, what if software could be generated for individual pharma supply planners, as their planning needs evolve day by day?

I agree with Andrew Ng's recent observation that "six months ago, coding agents were a novelty. They still frequently fail to deliver, but I find that they’re now working well enough that they might be genuinely useful to more and more people!".

Lately, I've started to see innovation in guardrails which could improve AI significantly for code generation. Initiatives like AlphaCodium are developing ways to ensure that Large Language Models (LLM) produce code that passes tests provided by the end users. The key is to orchestrate AI in an "agentic workflow". It’s called AlphaCodium and it’s not a simple LLM, it’s a chain of LLMs where each one is given different instructions. Each of these so-called “agents” perform their task and pass their result to the next agent. It’s called a multi-agentic system and it performs its own internal feedback among the agents such that it iterates until the code it produces actually passes the tests. It even generates its own new tests but it always gives more attention to a failing test provided by the user since the generated tests too could be wrong. I think of the user-provided tests as the Golden Tests, similar to the Golden Units used in annotation projects where there are prelabeled items that are used to determine which annotators can be trusted.

Actually, this is an old idea applied in a new context. For a while, I, along with I estimate 5-10% of developers, have been employing software development methods that start with a test. These methods, variously called Test-Driven Development (TDD) or Behavior-Driven Development (BDD) ensure that code written by human developers passes a predefined test. The innovation is to write the test and then give it to AI rather than a human. If the test passes, it doesn't matter if it was written by a human or an AI.

AlphaCodium is taking this to the next level though because it also generates new tests. This ensures all corner cases are tested. It always prioritizes the tests provided by the end user, which helps rule out the possibility a generated test is hallucinated. It combines a code coverage tool with AI to generate tests. This ensures all corner cases are tested.

I did an experiment with part of AlphaCodium, the part that generates tests called cover-agent. I was able to get it to generate 33 new tests for about $15 worth of OpenAI credits over a period of about 2 hours. That's relatively inexpensive and fast compared to me writing that code. And it freed me up to work on other things.  Here's a demo of it creating one simple test.

That said, the next steps for AlphaCodium to deliver on this idea of Personalized Software are a bunch of usability improvements.

First, users wouldn't want to have to specify all their tests in advance. Someone has already figured out a best practice, Bob Martin, who says to write a single test to start. I think the way a user will want to use this system is iteratively. It's just too hard to think of all the test cases up front. Professor Eric von Hippel at MIT calls this "sticky information". Iteration can unstick the user's need information. Start by getting the user to express something about their problem, then collaboratively write an example test showing inputs and expected outputs. Then generate some code, either with AI or a human software developer. And let the user try it so they can discover what information they forgot to specify. The need for iteration has been learned in the past through the development of the Agile software development methodology which welcomes late-breaking requirements. Until people evolve to unstick what's in their minds by thinking more clearly and communicating more clearly, the need for iteration is here to stay. So I fundamentally believe code AI generation needs to be an iterative process with humans in the loop to provide feedback.

Second, the inputs to our software are very complex, big JSON files representing initial inventories, demand forecasts, recipes, and so on. I was not able to use cover-agent for this complex inputs because cover-agent tended to hallucinate values that were incorrect. An improvement would be to allow the user to provide specifications for algorithms inputs, for example a JSON schema. AlphaCodium was developed for inputs that are simple to specify such as coordinates and a radius. An industrial-strength code generator would need a way to generate realistic complex algorithmic inputs.

Third, although I was able to get cover-agent to generate 33 tests on my utility code, getting it to generate code that spans across big modules of code is another story. I experienced several kinds of errors in trying to make this work, such as ModuleNotFoundError. This is an important problem because utility code is relatively simple and simple code tends not to break once it's written. It's like Kent C. Dodds said, "Write tests. Not too many. Mostly integration." because those are the valuable tests which will actually protect against bugs in a big project.

Fourth, although AlphaCodium has shown some progress toward generating code for algorithms, I'm eager to see how I might be able to use it generate code for entire systems. Instead of providing a test as input, could I provide it a drawing of a user interface and have it infer the use case and desired interactions? It may be quite some time before we get there, but ultimately users interact with a user interface, so I think it's important to keep this in mind as a north star.

In conclusion, I'm hopeful that AI will allow ABC-Plan to help extend our commitment to optimizing inventory not only for the pharma industry, but for the unique requirements of specific pharma companies or even specific planners as their planning needs change day by day.