Runtime AI

Last week, I wrote about my experience using the latest AI coding tools.  And several weeks ago I read this piece by Ohad Eder-Pressman re his experience in creating an entirely AI app.   The key idea that he is discussing is this:

Picture visiting a website where nothing exists until you arrive. No code waiting on a server. No routes defined. When the page loads, an AI generates the interface on the spot. You click a button, but there's no event handler. The AI sees the click and decides what should happen. You submit a form to /api/todos. That endpoint doesn't exist, has never existed. But the AI receives the request and responds with exactly what a todo API should return.

The model isn't a tool that helps build the app. The model IS the app.

And he is onto something.  The current AI coding tools are design-time tools, and this is great, but it is timid approach.  Pushing AI into the runtime of an app could replace a large part of the typical app architecture — there is an opportunity to simplify app design, provide greater functionality, and stop the endless dorking around with JavaScript frameworks, CSS, and other app minutiae.  I’ve been working on a couple of apps that serve to illustrate.

Blogs and AI

A blog is a simple app.   A database of posts.  Some schema and hierarchy conventions.  Some views of the posts, informed by schema and hierarchy conventions — a timeline view, category view, etc.   And some rendering code that is a mishmash of JavaScript frameworks, CSS, HTML, and more.  

Historically, I have created my blog using various blog tools, including Blogger, Movable Type, WordPress, Jekyll, and Hugo, among others.  All these tools know what a blog is and have a lot of embedded knowledge about blog schemas, blog views, the blog edit cycle, etc.  These tools start out simple but grow in complexity as they add more publishing features, design customization, etc.

Now, today, with design-time AI tools like Claude Code, I can walk away from all those tools and just tell Claude the kind of data I have and the kind of views I want. Claude will take care of all the schema munging, JS wrangling, and CSS hoohah and deliver a blog.   Which is great!  I no longer have the time in my life to consider the intricacies of JS frameworks or the complexities of CSS version 5, or any other web development topics.  

But I still end up with a codebase implementing a fixed blog view on my data.  It is easier to maintain now (maybe! The regression behavior of AI-generated code seems likely to be interesting.)  But it is still a fixed set of views.   What if I want to see the 100 books I love the most, or how my genre consumption has changed over time, or a comparison of what I was reading about in 2005 vs 2025?  Or what if I want to ask, “Will I like this new book that came out in 2025 based on my reading history,” or “What books have I liked that have not stood the test of time”?  With design-time AI tools, I will need to envision all these queries and instruct the AI tool to create the appropriate views.  And then redeploy the blog and fix any regression errors.  And now I will have new code deployed with a new, larger set of fixed views.  Rinse and repeat as I think of new ideas — which may be views that I use once, and never again, but now they are in the code and must be maintained.

I don’t want an AI-generated code base to maintain — I want to get out of the business of writing and maintaining blog code altogether.  What I want is a pure AI runtime with basically no code from me.  I should be able to ask my blog any question about its content and get a great view of the blog in return.  I should not have to imagine all my possible queries and views at design time — the AI engine should handle that on the fly.  IDE usage could largely fade away, or be very limited to selecting the AI tool and tweaking the system prompt.

It would be expensive to service every single query to my blog URL with an AI-generated response, and no one would want to pay for this.  An AI blog would need to be smart about a) using a local model to parse requests and filter out the simple ones, and b) using a cache of common views so that they don’t have to be continually recalculated.  Most of my blog traffic will be in the form of a timeline view and an RSS feed — I don’t need to engage the full power of AI every single time.  

Smart Home and AI

I have tried several times to build a simple smart home app, including just this past week.   We’ve had problems getting the right temperature comfort level in all the rooms in the home, and we have had some thermostat problems and zoned heating system problems that have been vexing to chase down. During the swing seasons, when temperatures can fluctuate by 30°F or more in a day and even more over the week, maintaining consistent temperatures in some rooms can be a challenge.  I have several thermostats in the house, one for each of the forced air zones, and another for the radiant heat in the bathrooms. Some of the thermostats (Ecobees) have additional remote sensors. And I have several other temperature sensors scattered throughout the house.  I’d like to create a simple app that constantly collects temperature readings and then provides table views and time series plots, so I can understand the existing performance of the system, and share the data with service people when they come out.

Anyone who has spent any time with smart home apps knows that the smart home is a morass of standards and proprietary protocols.  Collecting all this info will require a bunch of different protocol and vendor-specific drivers, and then some code to map all the various schemas and hierarchies together, and then some code to implement the views.

A tool like Claude can figure out Matter and HomeKit and Threads and all the other smart home frameworks and standards, which is great, I don’t want to burn up gray matter on all that.  Design-time AI can save a lot of time.

Again, though — what if I come up with a new view on my home that I’d like to see?  I want a heatmap view, I want to compare outdoor temperatures to indoor temperatures, I want to integrate presence information, I want alerts if humidity spikes, I want to incorporate camera views, I want to integrate some new vendor/standard, I want to incorporate maintenance history, etc.  I don’t want to have to think of each of these new features, build them, test them, and deploy them.   I just want the AI to figure this out at runtime.

The big AI opportunity – displacing static app code

I’ve spent a lot of hours in my life learning JavaScript, JavaScript frameworks, HTML variants, CSS, CSS frameworks, build tools, backend frameworks, debugging, deployment, etc.   I don’t want to now add into that all that learning about agentic frameworks and the ins and outs of each.  For a large class of apps, I just want to hand the whole problem to AI and let it be the app.  

The examples above seem like pretty typical app patterns, and I suspect a large number of business apps follow these patterns. An AI runtime that eliminates all the time spent working with various frameworks and delivers greater flexibility seems like a generally useful thing.  This is beyond just embedding an agent in an app — it is handing the entire app over to AI with only the lightest direction, and letting the AI be the app, evolving the experience as usage dictates.

Not every app would benefit from this architecture.  The consequences of getting a less-than-perfect answer to the query “What books does John Ludwig love the most?” are pretty minor.   The consequences of getting a less-than-perfect answer to a medical query are higher, so maybe we don’t rush to jam AI into the runtime of some apps — but it is worth starting to experiment.

It is likely cost-prohibitive to utilize cloud-based AI tools today to handle every query — see for instance, Anthropic is trying to rein in Claude usage — it is too costly today to use cloud-based AI in continuous fashion.  We need runtime solutions that utilize both local AI engines and cloud AI services, working in tandem.   

Coming back to Ohad Eder-Pressman’s article I cited earlier:

We've been using AI to write code faster. Cursor and Claude Code build entire features from natural language. These tools are incredible, but they're still operating in the old paradigm: we build software, then we run it.

He is dead on.  As AI costs drop and speeds rise, and as local models get more efficient, we can hand off more and more of creation and operation of an app to AI. An exciting time to be in the software business.