Free Nemotron on Vercel: A Useful Worker, Not a Private Workspace

The free badge is an easy reason to try a model. With Nemotron 3.5 Lightning on Vercel, the small print is a better place to start.

Vercel added the NVIDIA-served model to AI Gateway on October 14. Its current model page says the free endpoint logs prompts and output to improve NVIDIA’s products and services. It is for trials, not production or business-critical systems, and users are told not to send personal, confidential or sensitive information. That is a more specific restriction than simply saying a provider might retain data.

There is still an interesting model behind the offer. Lightning is aimed at the repetitive work inside an agent: making tool calls, checking results and carrying out a well-defined part of a larger task. The useful question is whether it can do enough of that work reliably—not whether it can replace the most capable model in every conversation.

Why a smaller worker can matter

NVIDIA describes Lightning as a 30-billion-parameter mixture-of-experts model, with about three billion parameters active for each token. The distinction matters: activating less of a model reduces the computation needed for a step. It does not mean the entire model has the memory footprint of an ordinary three-billion-parameter model.

The family architecture helps explain the approach. In NVIDIA’s December 2025 introduction to Nemotron 3, Mamba layers carry sequence state, attention layers connect information across the input, and a router selects specialist experts. Different machinery handles different parts of the work instead of making every token take the same expensive path.

That older article is background, not the announcement of Lightning 3.5. NVIDIA introduced Lightning in August 2026. Its explanation emphasizes high-volume execution and multi-token prediction: drafting several possible next tokens and verifying them efficiently. A faster route through routine steps could make a whole workflow more responsive, provided the checking and recovery steps still work.

A large advertised context window does not change that qualification. Vercel lists a one-million-token capacity, tool calling and structured outputs. Those are supported features, not evidence that a model will find the right fact in a long document or make a safe decision about a tool. An output can be valid JSON and still contain the wrong answer.

The more useful results come with trade-offs

LangChain’s Srimanth Tangedipalli and Karan Singh tested a related proposition: start work on Nemotron 3.5 Lightning and escalate to a more expensive model when a judge detects trouble. Their August study used 145 controlled multi-step tasks, rather than an assortment of impressive chat responses.

The routed setup cost 74% less than their frontier-only baseline, but accuracy fell from 86% to 80%. The judging model also accounted for 21.2% of routed spending. In other words, deciding when to use the expensive model was itself a material expense. These are results from their particular paid-model experiment, not a forecast for the free Vercel endpoint or a promise of savings on another workload.

I find the accuracy gap more informative than the headline saving. Six percentage points may be tolerable in a reversible drafting task and unacceptable in a workflow that changes records or sends messages. A router needs an error budget and a way to stop; a cheap worker alone does not supply either.

Thoughtworks’ early-access team approached the model from another direction. Gustavo Lujan, Allen Roush and Andy Nolan describe adapting it for specific domains and testing the adapted versions on held-out questions with blind comparisons. They also investigated changing its writing behavior rather than fixing every answer with another prompt. Thoughtworks explicitly identifies itself as an early-access partner, so this is a useful first-hand engineering account, not an independent verdict on every deployment.

What interests me there is the method: give a model a narrower job, keep examples used for evaluation separate from training, and check whether the change actually helps. The fact that a specialized version works well on one team’s evaluation does not make the untouched public endpoint a domain expert.

Open weights and a hosted endpoint are different choices

The distinction is easy to lose in a model picker. Open weights give a team another way to deploy and customize a model. A hosted free route gives access to a service under that service’s current conditions. Choosing one does not give you the data controls of the other.

For a confidential internal corpus, that is not a footnote. A prompt can include retrieved passages, attached files and previous messages as well as the sentence someone just typed. A harmless request to summarize a document may therefore transfer much more than the user realizes. This particular free route should not receive that material.

A trial worth running

I would start with a public announcement and a small task: extract its launch date, availability conditions and source address. Then introduce an older announcement that conflicts with it. Does the model keep the dates separate, identify what changed and refuse to fill a missing fact with a plausible guess?

That is close to real editorial work, yet the result is easy to inspect. The same idea transfers to a tool-using assistant: begin with read-only operations, check its arguments and make a failed tool result part of the test. Do not give a trial model permission to publish simply because it produced one convincing draft.

Nemotron’s appeal, to me, is the possibility of a well-chosen worker inside a larger system. The free Vercel route makes that worker easier to examine. It does not make every input appropriate, every task safe, or the service suitable for production. Those decisions still belong to the person building the workflow.

Sources checked October 15, 2026. This article is an editorial assessment of published specifications and first-hand reports, not a hands-on benchmark of the Vercel endpoint.