Choosing an LLM: Closed-Source APIs, Open-Weight Models or On-Premises Deployment

When a company decides to adopt large language models (LLMs), the first question is often not “what will we use it for?” but “which provider?”. New models are released almost every month, leaderboard rankings keep shifting, and today's “strongest model” may be overtaken within a few months. For companies, what matters more than which model is strongest is choosing the right way to use it, keeping control of where data goes, and retaining the ability to switch models at any time.

This article covers the three main ways companies use LLMs and their pros and cons, the real gap between open-weight and closed-source models, the key points of data compliance, and why you shouldn't bet everything on a single model, followed by a step-by-step selection guide and answers to frequently asked questions.

What are the ways a company can use LLMs?

There are three main ways: calling a closed-source model's API, using open-weight models, and on-premises or private deployment. The fundamental difference between them is not the model itself, but who runs the model, where the data goes, and who is responsible when something goes wrong.

Calling a closed-source model API. An API (application programming interface) can be thought of as a “remote service window”: the company sends questions and materials to the model provider, which runs the model on its own servers and returns the results. GPT, Claude, Gemini and others offer this kind of service, usually billed by usage, and the model's internal parameters (its “weights”) are not made public.

Using open-weight models. The weights of open-weight models can be downloaded, as with some models in the Llama, Qwen, DeepSeek and Mistral families. A company can deploy them on cloud servers it rents (self-hosting), or use a managed service from a third-party platform. Note that “open-weight” does not mean unrestricted; licenses vary widely. For example, Meta's Llama 4 Community License requires licensees with more than 700 million monthly active users to request a separate license from Meta, and requires “Built with Llama” to be displayed prominently on related products or websites (source).

On-premises or private deployment. The model is deployed in the company's own data center or a dedicated environment, and data never leaves the corporate network. This approach usually relies on open-weight models and offers the most control, but responsibility for hardware, operations and security falls entirely on the company.

The table below compares the main differences between the three approaches:

The table reflects typical cases; the same type of solution can vary considerably between providers.

For example, a distribution company wants to use AI to automatically extract amounts, tax IDs and due dates from supplier invoices. During the pilot, invoice volumes are small and calling an API directly is the simplest option. Once the solution is rolled out across the group, with high and stable volumes and strict requirements against data leaving the company, a self-hosted open-weight model may have the edge on cost and compliance. How you use LLMs is not a one-time choice; it should evolve with the scale of the business and its data requirements.

How big is the gap between open-weight and closed-source models?

Open-weight models have improved rapidly over the past two years. The gap has narrowed from a clear generational lead to single-digit percentages, but it has not disappeared, and it fluctuates.

The AI Index 2025, published in April 2025 by the Stanford Institute for Human-Centered Artificial Intelligence (Stanford HAI), found that the performance gap between open-weight and closed models on some benchmarks narrowed from 8% to 1.7% in a single year; meanwhile, the inference cost of a model performing at GPT-3.5 level fell more than 280-fold between November 2022 and October 2024 (source).

However, the latest edition, published in April 2026, shows that the gap widened again in 2025: as of March 2026, the top closed model on the Arena leaderboard led the top open model by 3.3%, compared with just 0.5% in August 2024, and six of the top ten models on the leaderboard were closed. The report also notes that models from several US and Chinese developers sit in the top tier with closely matched scores, and that competition is shifting toward cost, reliability and domain-specific performance (source).

These findings hold three lessons for companies:

• Don't treat rankings as a long-term verdict. The same report notes that US and Chinese models have traded the lead several times since early 2025. Today's number one may not be next year's.
• Leaderboard scores are only a reference point. The report also cautions that widely used benchmarks have reliability problems; one study found that up to 42% of the questions in some benchmarks were invalid (source). A few percentage points on a general leaderboard may not carry over to your specific business.
• Many tasks don't need a frontier model. For tasks such as ticket classification, extracting data from documents and summarizing internal documents, a solution that is capable enough, cheaper and offers more control over data is often a better fit than “the strongest” one.

For example, a retail chain uses a model to automatically sort in-store customer feedback into categories such as “product quality”, “service attitude” and “price”. The rules for this task are clear, and a mid-sized open-weight model may be perfectly adequate; analyzing complex supplier contracts for the procurement department, on the other hand, may justify a more capable model. Whether a model is good enough should be measured on your own data, not read off a leaderboard.

What should you watch for on data compliance when choosing an LLM?

Many companies compare only capability and price when selecting a model, and overlook where their data goes. When choosing a model, contract terms and data flows matter as much as model capability. The three areas below are the ones we are asked about most often; how they apply to your company should be confirmed with legal counsel.

The EU General Data Protection Regulation (GDPR). If your business involves personal data from the EU, two points deserve particular attention. First, Article 28 of the GDPR provides that when personal data is processed on your behalf, you may only use processors that provide sufficient guarantees, and the processing relationship must be governed by a contract or other binding legal act. Second, Chapter V (from Article 44 onward) provides that transfers of personal data outside the EU must meet the conditions set out in that chapter; in the absence of an “adequacy decision”, appropriate safeguards are required, such as standard data protection clauses (source).

China's cross-border data rules. If a company collects data in China and the model's servers are located abroad, sending data to the model may constitute a cross-border data transfer. The Provisions on Promoting and Regulating Cross-Border Data Flows, brought into effect by the Cyberspace Administration of China in March 2024, distinguish several scenarios. Data processors other than critical information infrastructure operators that, since 1 January of the current year, have cumulatively provided personal information (excluding sensitive personal information) of fewer than 100,000 people abroad are exempt from a security assessment, standard contract or certification. Those that have provided personal information of 100,000 or more but fewer than 1 million people (or sensitive personal information of fewer than 10,000 people) must sign a standard contract for the export of personal information or obtain personal information protection certification. Those that have provided personal information of 1 million or more people or sensitive personal information of 10,000 or more people, as well as those involving important data, must apply for a data export security assessment (source). For example, sending the purchase records of members nationwide to an overseas model for analysis would first require assessing whether any of these scenarios applies.

Provider data use terms. The same provider may apply entirely different terms to different versions of its service. Take Google's Gemini API Additional Terms of Service: for unpaid services, Google uses submitted content and generated responses to improve its products, human reviewers may read this content, and the terms explicitly warn against submitting sensitive, confidential or personal information; for paid services, Google does not use this content to improve its products, the data is processed under the data processing addendum, and it is retained only for a limited period to prevent abuse and meet legal requirements (source). OpenAI likewise states on its enterprise privacy commitments page that it does not train models on enterprise business data (source). This means that when employees process company materials with free personal accounts, entirely different data rules may apply than to the services the company has formally procured.

Before signing, confirm at least the following with the provider:

• In which country or region data is stored and processed;
• Whether inputs and outputs will be used to train or improve models;
• How long data is retained, and whether deletion can be requested;
• Whether there is human review, and where the reviewers are located;
• Whether other subprocessors are used, and whether you will be notified in advance of any changes.

Why shouldn't you bet everything on one model?

Tying all your business to a single model may look simple, but in practice it hands cost, capability and risk to a single provider. The ability to switch models should be a requirement from day one, not a fix applied after problems arise.

Companies are already moving in this direction. In a survey published in June 2025, the investment firm a16z interviewed 100 enterprise CIOs across 15 industries: 37% were already using 5 or more models, up from 29% the previous year. Companies source models from multiple providers mainly because different models excel at different tasks, and also to avoid lock-in to a single vendor. But the survey also found that switching costs are rising: teams have tuned prompts and guardrails for specific models, and in multi-step agentic workflows, changing one step can affect several downstream steps (source). The survey's sample is only 100 people and reflects an investor's perspective, so its conclusions should be treated as indicative.

A common multi-model approach is “routing”: automatically sending each request to the most suitable model based on the nature of the task.

• Route by difficulty. Simple, high-volume tasks such as classification and data extraction go to fast, low-cost models; complex analysis and writing go to more capable models.
• Route by data sensitivity. Requests containing sensitive information such as ID numbers, bank accounts or employee salaries are handled only by on-premises or privately deployed models.
• Route for availability. If a provider suffers an outage or raises prices sharply, traffic can be switched to a backup model so the business keeps running.

For example, a customer service system can let a small model handle common questions such as “track my delivery” or “change my address”, pass complaint escalations and refund disputes to a more capable model, and process tickets containing customers' ID documents only on an on-premises model.

To make routing work in practice, three things are needed on the technical side: a unified access layer between business systems and models, so that no system connects directly to a particular provider; minimal reliance on any one provider's proprietary features; and keeping prompts, evaluation sets and business data in your own hands, so that you have a sound basis for comparison when switching models.

How to choose the right LLM for your business: a step-by-step guide

Selection should start from business tasks, not a list of models. First be clear about the problem you need to solve and the risks you can accept, then decide which model to use and how.

Step 1: Start from tasks and list specific requirements. For each task, write down the inputs, outputs, processing frequency, cost of errors and sensitivity of the data involved. For example: “Extract amounts and tax IDs from supplier invoices daily; errors lead to incorrect payments; the data includes supplier bank accounts.”

Step 2: Build an evaluation set from real business data. Select a representative sample from actual operations, anonymize it and write reference answers. Public leaderboards can only help you narrow down the candidates; what ultimately counts is how the candidate models perform on your own evaluation set.

Step 3: Define the options by data sensitivity. First determine which data can be sent to external APIs, which involves cross-border transfer and which must stay on-premises, then rule out solutions that don't meet these requirements. A solution that fails on compliance should not make the shortlist, however capable it is.

Step 4: Calculate total cost of ownership, not just the unit price. For API solutions, estimate costs as usage grows; for self-hosting and on-premises deployment, include compute or hardware, power, operations staff, security and upgrades. Both types of solution also need to account for system integration, performance evaluation and human review. Because model prices fall quickly, cost estimates should be updated regularly.

Step 5: Run a small pilot comparing two or three candidate solutions. In the same business scenario, compare accuracy, response time, cost and stability using the same evaluation set and real traffic, and gather feedback from frontline users.

Step 6: Plan your exit path in advance. In the contract, cover data export and deletion, advance notice of price changes and liability for service outages; on the technical side, decouple from specific models through a unified access layer. That way, changing models in the future is a switch, not a rebuild.

Step 7: Reassess regularly. Retest with the same evaluation set at regular intervals or whenever a major new model is released, and decide whether to adjust your model mix in light of cost changes.

Frequently asked questions about choosing an LLM

Do SMEs need to deploy LLMs on-premises? In most cases there is no need to start with on-premises deployment. It requires a large upfront investment and dedicated technical staff to maintain it. You can start with a paid API service with clear data terms, and consider private deployment only when highly sensitive data or explicit localization requirements are involved.

Are open-weight models free? Downloading the weights is usually free, but running the model requires compute, operations and staff, and at low volumes it may not be cheaper than an API. In addition, license conditions vary from one provider to another, so read them carefully before commercial use.

What are the risks of employees using free AI tools for work? The data terms of free versions and paid enterprise versions can be completely different; content submitted to free versions may be used to improve models and may even be reviewed by humans. Companies should specify which materials may be entered into external AI tools, and provide employees with vetted, approved tools.

Models evolve so fast. Will whatever I choose now soon be outdated? New models will keep appearing, but that is nothing to fear. Once you have your own evaluation set and a unified access layer, you can test each new model as it appears and switch when it makes sense, turning model selection from a one-off bet into a process of continuous improvement.

References

• The 2025 AI Index Report, Stanford HAI, April 2025
• The 2026 AI Index Report: Technical Performance, Stanford HAI, April 2026
• Llama 4 Community License Agreement, Meta, April 2025
• Regulation (EU) 2016/679 (General Data Protection Regulation), EUR-Lex, April 2016
• Provisions on Promoting and Regulating Cross-Border Data Flows, Cyberspace Administration of China, March 2024
• Gemini API Additional Terms of Service, Google, updated April 2026
• Enterprise privacy: Our commitments, OpenAI, accessed November 2026
• How 100 Enterprise CIOs Are Building and Buying Gen AI in 2025, Andreessen Horowitz (a16z), June 2025