AI App Development

What an AI application is and when it is worth building

An AI application is ordinary software (interface, backend, database, permissions) in which a language model does the core job of the product: reading documents, answering questions, classifying, generating or taking steps in a process. Building one makes sense when that job is specific to your data or process and an off-the-shelf tool cannot handle it.

Most of the code in such an app is still classic engineering. The model is one layer, but the most expensive and least predictable one: it responds with latency, costs money on every call and is sometimes wrong. That is the main difference from a regular software project: from week one you measure answer quality and model cost, not just whether a feature works.

Build or buy

The 2025 MIT NANDA report, as covered by Fortune, found that deployments based on tools bought from specialised vendors reached production in about 67% of cases, while purely internal builds succeeded roughly a third of the time. I read that as an argument to build only where a ready-made solution genuinely does not fit, not because custom code sounds better.

• Build when AI is the core of the value of the product you sell, or when your process and data are unusual enough that a SaaS tool would force you to change how you work.
• Build when the data cannot go to a third-party tool or you need full ownership of the code.
• Buy when a tool already covers most of your needs and the gaps are not critical.
• Start with automation, not an app when you need to connect existing systems and nobody needs a new interface.

AI app architecture – model APIs, open-weight models, RAG and evals

A typical AI application has four layers: the interface, a backend with logic and data, an AI layer (model, document retrieval, tools), and evaluation plus monitoring that tell you whether the whole thing works. I start with the simplest AI layer that solves the task and add complexity only when tests show it is needed.

Model APIs or open-weight models

Commercial model APIs (OpenAI, Anthropic, Google) give you the fastest start and the strongest models, and you pay per use. Open-weight models on your own infrastructure give full control over data, but you take on GPU servers, updates and re-testing after every model version change. In most projects I start with an API under a data processing agreement and add a local model only for steps that touch the most sensitive data. How to compare them on your own data is covered in how to choose an LLM.

RAG: answers grounded in your documents

RAG (Retrieval-Augmented Generation) first retrieves matching document chunks and only then lets the model write the answer. Quality depends mostly on retrieval: in its Contextual Retrieval write-up (2024), Anthropic showed that adding context to chunks and combining vector search with BM25 cut failed retrievals by 49%, and by 67% with reranking, without changing the model. A production example is the quoting engine built on RAG over 70 historical cost estimates in the Konkret case study.

An agent only when a workflow is not enough

Anthropic's "Building effective agents" (2024) recommends finding the simplest solution possible and only increasing complexity when needed. In apps, that usually means a fixed path with one or two model calls beats an agent that picks its own tools: it is cheaper, faster and easier to test.

AI MVP scope – cutting the build down to one hypothesis

An AI MVP is the smallest working version of the product that tests one hypothesis with real users: that the AI solves their problem well enough for them to use it and pay for it. Everything that does not serve that hypothesis waits for the next version.

The discipline comes from data. In its 2026 analysis, CB Insights reviewed 431 VC-backed startups that shut down since 2023. Poor product-market fit was cited in 43% of cases and running out of capital in 70%, with the authors noting that running out of money is usually the final symptom rather than the root cause. Code is rarely the biggest risk in an MVP.

What stays in the MVP and what goes

• Stays: one product loop from user input to AI output, sign-in, a database, behaviour analytics and a test set to score the answers.
• Stays: model cost per active user, because in AI products that number decides whether the business model works.
• Goes: an admin panel with many roles, "just in case" integrations, multiple models, polished onboarding and features nobody has asked for yet.
• Goes: an agent where a fixed path with a single model call can test the hypothesis.

Scope, timeline and deliverables are on the AI MVP development page.

How AI app development works step by step, and how long it takes

I usually build an AI MVP in 4–8 weeks; a full application takes longer, depending on the number of modules, integrations and security requirements. I work in two-week sprints, and from the first sprint you have a working version to click through.

1. Workshop and scope (a few days). Problem, user, data, hypothesis and definition of success. This produces the timeline, the quote and the stack.
2. Test set before code. I collect sample inputs and expected outputs, including hard cases. These later score prompts and models.
3. Foundation and AI loop. Data architecture, model integration behind an abstraction layer and the first working path from input to output.
4. Interface and full flow. Authentication, database, error and loading states, and a staging version for testing.
5. Testing, hardening and launch. Automated tests, evals, attack tests on the AI layer, monitoring and production deployment.
6. Handover. Code in your repository, documentation, a runbook and a session with the team taking over.

The code itself is rarely what takes longest. Delays usually come from missing sample data at the start, scope decisions that wait for weeks, and integrations with systems whose API nobody has documented. That is why I ask for sample data and one decision-maker on the client side before we start.

How much an AI app costs – build and running costs

The cost of an AI app is a one-off build plus running costs: infrastructure, model usage and maintenance. With me, an MVP starts from €4,500, an AI backend from €3,500 and a custom web application from €8,000.

Infrastructure (Vercel, AWS, GCP) is typically €50–500 a month depending on traffic. Model fees depend on the number of calls and context length, so I estimate them as a cost per active user or per operation, not as a single monthly figure.

Levers that bring model costs down

• Batch mode. OpenAI's Batch API documentation lists a 50% discount versus synchronous calls in exchange for results within 24 hours. Good for reports, classification and overnight processing.
• Prompt caching. According to Anthropic's documentation, a cache read costs 10% of the base input token price, and less on some newer models. Worth it for long, repeated instructions and documents.
• Matching the model to the task. A smaller model for classification and extraction, a stronger one only for the steps that need it.
• Shorter context. Less unnecessary history and better RAG retrieval mean fewer tokens on every call.

More techniques with examples are in AI API cost optimisation.

How to measure AI app quality – evals and product metrics

I measure AI app quality on two levels: evals check model answers against a fixed test set, and product metrics show whether people use the app and whether the costs work. Without evals, every prompt or model change is guesswork.

Anthropic's documentation on evaluations recommends starting from specific, measurable success criteria, building tests that mirror the real task distribution including edge cases, and favouring automated grading: exact match, code-based checks or grading by a second model. More automatically graded tests beat a handful of hand-graded ones.

What I measure in every app

• Answer quality on the test set, run in CI on every change to the prompt, model or retrieval.
• Model cost per user or per operation, plus response time.
• User activation and retention, or for internal tools, task time compared with the previous process.
• Production errors: failed calls, timeouts and cases the system handed over to a human.

The second-model grading method is described in LLM evaluation with LLM-as-a-judge. The same pattern powers AnswerLyzer, my system in which a separate model grades other models' answers against defined criteria.

AI application security – the OWASP LLM Top 10 in practice

An AI app carries every risk of a normal web app plus a few new ones, collected in the OWASP Top 10 for LLM Applications 2025. First on the list is prompt injection (LLM01): a malicious instruction hidden in content the model reads can change its behaviour.

The OWASP list also covers, among others, sensitive information disclosure (LLM02), improper output handling (LLM05), excessive agency (LLM06), system prompt leakage (LLM07), vector and embedding weaknesses (LLM08) and unbounded consumption (LLM10). In the apps I build, LLM01, LLM05, LLM06 and LLM10 take the most work.

• Model output is untrusted data. I validate format and ranges before it reaches the database, the HTML or another system.
• User permissions apply in RAG too. Retrieval returns only documents the person is allowed to see.
• Least privilege for tools the model can call, and human approval for irreversible actions.
• Cost and call limits per user and per hour, so a bug or abuse does not turn into an invoice.
• Fallback to another provider's model. On 10 June 2025 the OpenAI API returned elevated error rates for several hours; apps built on a single model with no backup plan were effectively down.

Defences against prompt injection are covered in prompt injection and LLM application security. If the app processes personal data, GDPR applies as well: a processing agreement with the model provider, data minimisation in prompts and often a DPIA.

Software house, freelancer or in-house team – choosing who builds it

Choose a software house for large scope, several parallel workstreams and the need to replace people. Choose a single engineer when one pair of hands can carry the project and you want short communication lines. Build an in-house team when the product is validated and will be developed for years.

Questions to ask any provider

• How will you measure AI answer quality, and will I get the test set?
• What will the model cost per user, and how did you calculate it?
• What happens when the model provider has an outage or raises prices?
• Will the code, repository and cloud accounts be in my name from day one?
• How do you protect the app against prompt injection and data leaks?
• Who takes over after launch, and what do I get at handover (docs, runbook, tests)?

Where your data goes and how to keep it under control is covered in AI data security.

Common mistakes in AI app development

Most failed AI projects do not fail because of a weak model but because of unclear value, unprepared data and costs nobody calculated. Gartner (2024) predicted that at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025, citing poor data quality, inadequate risk controls, escalating costs and unclear business value.

1. An AI chat bolted onto the app with no job. An AI feature needs a clearly described task and a measure of success.
2. Judging quality on a few good examples. A demo on easy cases instead of a test set with exceptions.
3. Model cost calculated after the first big invoice. The most expensive model for every task and no limits.
4. One model provider with no abstraction layer. Switching models means rewriting logic, and a provider outage takes the product down.
5. A full product instead of an MVP. Months of feature work before anyone checks whether users want the core.
6. Code without tests or documentation. It works on the author's machine but cannot be extended or handed over.

If you already have a prototype that cannot handle traffic or costs, I usually start with AI systems programming: model routing, caching, queues and monitoring. When the product is validated and needs a full version, the next step is a custom AI web application.