Reaigent / Articles

What comes after the chatbot?

From agents using software to robots acting in the physical world, AI is giving us more to think about than answers in a chat window. Here's where I think its interactions with us, and the world around us, are heading.

There's a lot happening in AI. Scientific discovery, improvements in reasoning, how models are trained, and the applications being built around them could each fill their own article.

The part I'm interested in here is how we communicate what we want, how AI turns that into action, and what that interaction might look like in the future.

Looking back at Google's "Attention Is All You Need" paper from 2017, it's interesting to see just how much followed from it. Conversational AI was already a thing, but the Transformer architecture changed the trajectory. BERT helped push language understanding forward. Semantic search made finding relevant information easier. RAG connected retrieval to generative models, allowing them to produce answers using retrieved material. Different pieces of a much bigger picture.

Then GPT-3 came out in 2020. To me, that was when the potential really became obvious. It was revolutionary, and most people outside the industry didn't know much about it yet.

ChatGPT only came out two and a half years later, running on the GPT-3.5 series. Instruction tuning and human feedback helped turn those language capabilities into something people could actually have a useful conversation with. It quickly became what was described at the time as the fastest-growing consumer app in history. (Reuters, February 2023)

Those who were already using OpenAI's APIs and incorporating these models into applications had a head start on seeing where this could go.

One of the clearest signs was when OpenAI added function calling to its API in 2023. A model could interpret a human's request and turn it into a structured instruction for software to execute. Tool use became a bridge between conversation and action.

And looking at what they've been building, I think the AI companies had a much clearer idea of where they were heading than the early products let on. The pieces are starting to fit together. An assistant that understands what you mean, has the relevant context, and can get things done across the systems around you. Something a little closer to the "Jarvis" we'd joke about having one day.

Natural language was becoming a practical interface to machines. And the models kept getting better through more training, better post-training, and more computation spent working through a problem.

So what else were AI companies pursuing? AI interfacing with systems: apps, databases, computers, other AIs.

And here we are today:

  • MCP: a standard protocol for connecting AI applications to a system's tools and data.

  • A2A: a protocol for agents to communicate, coordinate, and delegate work.

  • Computer use and CLI access: agents interacting with software through screens, keyboards, and terminal commands, within the permissions of their environment.


Which leads us to the question... What's next?

Here's my guess:

There's a lot happening in AI. Scientific discovery, improvements in reasoning, how models are trained, and the applications being built around them could each fill their own article.

The part I'm interested in here is how we communicate what we want, how AI turns that into action, and what that interaction might look like in the future.

Looking back at Google's "Attention Is All You Need" paper from 2017, it's interesting to see just how much followed from it. Conversational AI was already a thing, but the Transformer architecture changed the trajectory. BERT helped push language understanding forward. Semantic search made finding relevant information easier. RAG connected retrieval to generative models, allowing them to produce answers using retrieved material. Different pieces of a much bigger picture.

Then GPT-3 came out in 2020. To me, that was when the potential really became obvious. It was revolutionary, and most people outside the industry didn't know much about it yet.

ChatGPT only came out two and a half years later, running on the GPT-3.5 series. Instruction tuning and human feedback helped turn those language capabilities into something people could actually have a useful conversation with. It quickly became what was described at the time as the fastest-growing consumer app in history. (Reuters, February 2023)

Those who were already using OpenAI's APIs and incorporating these models into applications had a head start on seeing where this could go.

One of the clearest signs was when OpenAI added function calling to its API in 2023. A model could interpret a human's request and turn it into a structured instruction for software to execute. Tool use became a bridge between conversation and action.

And looking at what they've been building, I think the AI companies had a much clearer idea of where they were heading than the early products let on. The pieces are starting to fit together. An assistant that understands what you mean, has the relevant context, and can get things done across the systems around you. Something a little closer to the "Jarvis" we'd joke about having one day.

Natural language was becoming a practical interface to machines. And the models kept getting better through more training, better post-training, and more computation spent working through a problem.

So what else were AI companies pursuing? AI interfacing with systems: apps, databases, computers, other AIs.

And here we are today:

  • MCP: a standard protocol for connecting AI applications to a system's tools and data.

  • A2A: a protocol for agents to communicate, coordinate, and delegate work.

  • Computer use and CLI access: agents interacting with software through screens, keyboards, and terminal commands, within the permissions of their environment.


Which leads us to the question... What's next?

Here's my guess:

Robotics

Robotics

We can already see it happening, but there's still a long way to go when it comes to machines that can handle a wide range of everyday tasks. Take Teslas. A car that uses cameras and AI to navigate roads and control its movement is a pretty tangible example of this direction. Tesla's consumer FSD is still supervised driver assistance, but it gives people a glimpse of software interpreting the physical world and acting in it.

Gemini Robotics performing dexterous manipulation tasks.

Gemini Robotics performing dexterous manipulation tasks. Source: Gemini Robotics Team, Google DeepMind, Figure 15. Licensed under CC BY 4.0.

AI-generated illustration of a Tesla-like electric car.
AI-generated illustration.

What I'm watching is how far these systems can move beyond a specific task like driving: understanding a request, figuring out what needs to happen, and carrying it out in a changing physical environment. Models that connect vision, language, and physical actions are already being developed. (Gemini Robotics) The interesting question is how broadly and reliably those capabilities will translate into everyday use.

We can already see it happening, but there's still a long way to go when it comes to machines that can handle a wide range of everyday tasks. Take Teslas. A car that uses cameras and AI to navigate roads and control its movement is a pretty tangible example of this direction. Tesla's consumer FSD is still supervised driver assistance, but it gives people a glimpse of software interpreting the physical world and acting in it.

AI-generated illustration of a Tesla-like electric car.
AI-generated illustration.

What I'm watching is how far these systems can move beyond a specific task like driving: understanding a request, figuring out what needs to happen, and carrying it out in a changing physical environment. Models that connect vision, language, and physical actions are already being developed. (Gemini Robotics) The interesting question is how broadly and reliably those capabilities will translate into everyday use.

Gemini Robotics performing dexterous manipulation tasks.

Gemini Robotics performing dexterous manipulation tasks. Source: Gemini Robotics Team, Google DeepMind, Figure 15. Licensed under CC BY 4.0.

AI-enhanced human-computer interaction (HCI)

AI-enhanced human-computer interaction (HCI)

This is one I'm especially interested in.

We're already beyond text. AI can speak, understand images, generate visuals within a conversation, and build interactive interfaces. But I think there's a lot more coming in how naturally all of that fits into an interaction.

Brain-computer interfaces (BCIs) could be the endgame here, with companies like Neuralink already enabling trial participants with paralysis to control computers using their thoughts. Whether we'll end up interacting with AI in this way on an everyday basis is another question.

For what's taking shape now and over the next few years, I'm putting my money on agent-driven user interfaces.

I imagine an interaction where the AI can express an idea through whatever makes the most sense at that moment. A diagram, a simulation, an interactive model, or something we can both work with while discussing.

We're already seeing early versions of this. Google's A2UI project, for example, lets agents generate interfaces suited to the current interaction.

Spatial panels and floating controls in Android XR.
Spatial panels and floating controls in Android XR. Source: Android Open Source Project, "Spatial UI." Licensed under CC BY 2.5.

How far can we take this? The interface becomes part of the agent's context. It knows what it has shown you, what you've changed, and what you're referring to. It can adapt the interface as the conversation develops. Imagine discussing a business idea and working through an interactive model together. You change an assumption, the projections update, and the agent understands the change and continues the discussion from there. Then extend that into AR and VR. There are already early versions of this too, with Gemini integrated into Android XR.

This is one I'm especially interested in.

We're already beyond text. AI can speak, understand images, generate visuals within a conversation, and build interactive interfaces. But I think there's a lot more coming in how naturally all of that fits into an interaction.

Brain-computer interfaces (BCIs) could be the endgame here, with companies like Neuralink already enabling trial participants with paralysis to control computers using their thoughts. Whether we'll end up interacting with AI in this way on an everyday basis is another question.

For what's taking shape now and over the next few years, I'm putting my money on agent-driven user interfaces.

I imagine an interaction where the AI can express an idea through whatever makes the most sense at that moment. A diagram, a simulation, an interactive model, or something we can both work with while discussing.

We're already seeing early versions of this. Google's A2UI project, for example, lets agents generate interfaces suited to the current interaction.

How far can we take this? The interface becomes part of the agent's context. It knows what it has shown you, what you've changed, and what you're referring to. It can adapt the interface as the conversation develops. Imagine discussing a business idea and working through an interactive model together. You change an assumption, the projections update, and the agent understands the change and continues the discussion from there. Then extend that into AR and VR. There are already early versions of this too, with Gemini integrated into Android XR.

Spatial panels and floating controls in Android XR.
Spatial panels and floating controls in Android XR. Source: Android Open Source Project, "Spatial UI." Licensed under CC BY 2.5.

What happens when these capabilities become part of everyday life?
What could that look and feel like?

Those are questions I'm eager to explore. For me, this is an interesting space to build in. As AI gets more capable of doing things, how we communicate, collaborate, and stay involved in the process becomes a bigger part of the product.

So, what are you putting your money on?


Join the discussion on LinkedIn · Follow Reaigent

What happens when these capabilities become part of everyday life?
What could that look and feel like?

Those are questions I'm eager to explore. For me, this is an interesting space to build in. As AI gets more capable of doing things, how we communicate, collaborate, and stay involved in the process becomes a bigger part of the product.

So, what are you putting your money on?


Join the discussion on LinkedIn · Follow Reaigent