LLM and GenAI future from a business perspective
I had the pleasure to share some time with customer facing roles.
This occasion reinforced my thoughts on the future of LLM and GenAI that I was asked to share.
This post is a more digestible review of the status of LLM and GenAI, without the specific inclination of my organization.
I like to start with a few ideas that I believe to be true.
LLM and GenAI are genuinely useful tools
They cannot THINK, but so what? They are still useful.
LLM will not solve problems in a creative and new way. But they will help us human to think, brainstorm, organise, and overall be more productive.
Their cost will tend to zero
Cost of tokens for LLM is trending down to zero.
While there will be a slightly better model today, the models of 6/12 months ago are basically commodity. Pick your vendor by price or convenience. But all the models are roughly equivalent.
Context window while increasing will stay limited
While the context size is keep increasing, it will eventually stop growing due to the limitations of the models and the hardware - and the associated costs.
The context will remain too limited to load entire organizations know how.
These ideas lead us to my main thesis.
Those who control, manage, and can work with organizations data will be the winner with LLM.
Not the providers of the models, not the provider of the hardware. But those that are able to fetch, manipulate and structure data in a way that LLMs can use them.
Model providers and hardware providers will do great for themselves - but the real gold is in getting access to, manipulating and structuring data in a way that LLM can leverage it.
The first candidate seems to be Apple and their new Apple Intelligence.
Thanks to their control over both hardware and software in the Apple ecosystem, they were able to exploit LLM in a very impressing way - we will still need to see if an impressive demo translates into an impressive product and revenue.
Apple Intelligence does indeed exploit LLM and GenAI, it is offered for free - clear sign that their cost is not a big barrier.
Their secret sauce is understanding what information is relevant for the task at hand, what prompt will be useful and feed those to an LLM.
Microsoft is in an even more interesting, and potentially profitable, position than Apple and Google.
Microsoft Sharepoint is the data platform for most business, and this put Microsoft in a very privileged position to just “Turn AI ON”.
You could imagine a lot of use cases that business will be willing to pay for. For instance, autocompletion and generation of documents or worksheet with information pulled directly from the source of truth of the company.
Or supporting the customer success department by automatically pulling the latest information.
All on top of the security and access control already provided by Sharepoint.
Making sure that the correct access policies are applied to data and documents that are used to feed LLM and GenAI will be of paramount importance. No organization will provide their employees access to LLM if no structured, auditable and safe access policies are in place.
But it is exactly in allowing employees to have access to LLM with data of the organization where most of the value lays.
Controlling, managing and structuring data
The complexity and the value of employ LLM and GenAI to streamline business operations lays in organize information in such a way that these tools can take care of the tedious tasks while leaving to people the creative and innovative tasks.
This is not yet been developed and industrialized for everybody to benefit.
I imagine an interesting development in the Software Engineers market. As soon as organization understand the need to structure data in the correct way to make them useful for LLM new jobs will be created.
Each system and each silo have its own little twist, making it difficult to re-use the same solution over and over.
A lot of integration work will need to get done.
On top of that, a lot of security and access control work will be needed to verify and auditing that information are exposed at the right level and that no internal information is leaked.