The question I get most from CTOs and IT heads in the Gulf is some version of "which model should we buy?" Claude or OpenAI. Gemini or an open model you run yourself. The licence is usually already half signed by the time they ask.

My honest answer is that there is no single best model. There is the model that does your task well enough, at a cost you can live with, inside your security rules. That answer is different for an invoice than it is for a customer email, and it changes every few months as new models come out.

So we don't pick a model from a leaderboard. We test a few of them on your work and let the results decide. Here is how that works.

Start from real cases, not a benchmark

Public benchmarks measure what the model makers chose to measure. They tell you very little about whether a model can read your supplier invoices, sort your support emails or check a claim against your policy.

We start with a set of your real cases, usually a few hundred, with the right answers attached. The right answer is whatever your best person on the team would have done. We keep the messy ones in on purpose: the crooked scans, the half Arabic, half English emails, the forms with a field missing. If a model only works on the clean cases, we want to find that out on day two, not after go-live.

Agree on what "good" means first

Before we run anything, we write down with you what a pass looks like for each case. For an invoice, that might be every field extracted correctly and the right cost centre picked. For an email, it might be the right category and a draft reply your team would send with light edits.

This step matters more than which models we test. If you can't agree on what a good answer is, no model will give you one.

Run the same cases through several models

We then run the same cases through a small shortlist. Usually that includes a model from Anthropic, one from OpenAI, often one from Google, and at least one open model that you could run on your own servers or in a cloud region you choose. Open models matter for companies that need data to stay in the country or want to avoid depending on one vendor.

Every model gets the same instructions and the same cases. We don't tune one and leave the others on defaults.

Score more than accuracy

A model that gets the most answers right is not always the right pick. Our scorecard has a few columns:

  • Accuracy: how many cases it got right, and which ones it got wrong.
  • Type of mistake: a wrong date on an invoice is not the same as a wrong amount. We weigh mistakes by what they would cost you.
  • Cost per task: what one finished task costs, including the human checks it still needs. I wrote more about this in Cost per task, not cost per token.
  • Speed: how long a task takes end to end, and whether that matters for this work.
  • Where the data goes: which region the model runs in, what the vendor's business terms say about your data, and whether that fits your security rules.

Laid out side by side, the choice is usually obvious. Sometimes the smaller, cheaper model is good enough. Sometimes only the largest one handles your edge cases. Sometimes no model is good enough yet, and we tell you that.

Why the answer differs by task

In practice, most companies end up with more than one model. One for reading documents, another for writing replies, maybe an open model for anything sensitive. That is not a problem as long as your system is built so you can swap a model out without rebuilding everything around it.

That is also why we keep the test cases. When a new model comes out, we rerun the same scorecard and you can see in a day whether switching is worth it. No sales pitch needed.

Why we can do this honestly

Skcript Applied Labs doesn't sell a model and isn't owned by a model company. We don't earn more if you pick one vendor over another. Our job is to find what works on your cases and get it running inside your tools.

If you are about to sign a model licence, or you already have one and aren't sure it was the right call, start a chat with us on the site, or book a call through the Applied Labs page. Bring one task and a set of real examples. We'll show you how the models compare on your work, not on someone else's benchmark.