ThoughtWorks moved Text to SQL to the Hold ring on its Technology Radar in November 2025, telling teams to stop pointing an LLM at a raw database and letting it write the query.[1] For most software, Hold is a shrug. For upstream operators it is the right call, and the reason has almost nothing to do with how good the model is.
The demo always works. You type “show me last month’s production for the Anderson wells” and a clean SQL statement comes back with a number attached. The problem is that the number is wrong often enough to matter, and there is no way to tell which times those are. That is a bad property for anything that touches an AFE, a JIB, or a reserve report.
What Text to SQL promised
The pitch is that everyone becomes an analyst. A landman, a reservoir engineer, or a comptroller types a question in plain English, an LLM turns it into SQL against your warehouse, and the answer appears. No ticket to the data team. No waiting three days for someone to write a query.
It is a good pitch because the bottleneck is real. Most mid-size operators do have a queue of ad hoc data requests that a handful of people service by hand. If natural language could reliably drain that queue, it would be worth a lot.
ThoughtWorks itself had Text to SQL in Trial as recently as April 2024. The move to Hold is not reflexive AI skepticism. It is a group that tried the technique in production and watched it fail in a specific way.
Where it actually fails
The failure is not syntax. Modern models write syntactically valid SQL almost every time. The failure is semantic: the query runs, returns rows, and answers a slightly different question than the one you asked. ThoughtWorks names the mechanism directly, that LLMs hallucinate because of limited schema or domain understanding, and that the non-deterministic output makes the errors hard to debug and audit.[1:1]
Two things make that worse than it sounds. First, a wrong SQL query does not throw an error. It returns a number, and a number always looks like an answer. Second, the model gives you a different query for the same question depending on phrasing, so you cannot even build a habit of trusting a particular output. The failure mode is confident, silent, and inconsistent. That is the worst combination for anything feeding a financial or regulatory process.
We have written before about where LLMs earn their place and where they quietly create errors in field ticket extraction. Text to SQL sits on the wrong side of that line for the same reason: the cost of a silent wrong answer is high, and the human checking it usually cannot see the mistake without redoing the work.
Why upstream data makes it worse
The standard argument for Text to SQL is that bigger, smarter models will close the gap. In upstream that argument is backwards, because the problem is not model quality. It is that the definitions live outside the schema entirely.
Consider what an LLM has to know to answer “what did the Anderson wells produce last month” correctly:
- Which physical wells are “the Anderson wells.” The lease name maps to different identifiers in the production system, the accounting system, and SCADA, and the API number is not a clean primary key. We spent a whole post on why a single master well table is hard precisely because of this.
- Whether “production” means gross wellhead volume, net of shrinkage, sales volume, or allocated volume. These are different numbers and they are all correct answers to different questions.
- Which allocation logic to apply when a measured battery volume gets pushed back to individual wells. That logic is a business rule, not a column.
- Whether the question wants working interest or net revenue interest, and whose interest. A comptroller and a reservoir engineer asking the “same” question want different math.
None of this is in the table definitions. A model looking at a schema sees a production table with a volume column and a well_id column. It does not know that your allocation model distributes a battery total across four wells by a twelve-month rolling factor, or that two of those wells are in a unit with a non-operated partner. So it guesses, plausibly, and you get a number that is off by the exact amount of the domain knowledge it did not have.
This is the same reason RAG on upstream data fails without preparation. The retrieval step, or the SQL generation step, is only as good as the semantics wrapped around the raw data. Dirty or undefined semantics in, confident wrong answers out.
What a governed semantic layer looks like
ThoughtWorks’s recommendation is to stop giving the model raw database access and put a governed semantic layer in between, naming dbt’s semantic layer and Cube as the tools.[1:2] The idea is old and boring, which is a point in its favor: define your metrics and entities once, centrally, so that “production” and “net revenue interest” mean exactly one thing no matter who or what is asking.
In dbt terms, that means defining semantic models and metrics rather than leaving the calculation to whoever writes the query. A stripped-down version for allocated production:
semantic_models:
- name: well_production
model: ref('fct_production')
entities:
- name: well
type: primary
expr: well_id
dimensions:
- name: production_date
type: time
type_params:
time_granularity: day
measures:
- name: allocated_gas_mcf
agg: sum
description: >
Battery-measured gas allocated to well level using the
rolling 12-month allocation factor. Net of shrinkage.
Excludes flared and fuel gas.
metrics:
- name: net_gas_production
label: Net Gas Production (Mcf)
type: simple
type_params:
measure: allocated_gas_mcf
The value is not the YAML. It is that the definition of net_gas_production (allocated how, net of what, excluding what) now lives in one versioned, reviewed place. The allocation logic is encoded once. Shrinkage handling is encoded once. When the reservoir team and accounting disagree about a number, they argue about the metric definition in a pull request, not about whose spreadsheet is right.
This is the same discipline we make the case for in dbt for OT and SCADA data. Encode the domain logic in a tested, versioned transformation layer instead of re-deriving it in every tool. A semantic layer is that idea pointed at metrics and entities specifically.
Now both a human and a model query the same defined metrics instead of raw tables. Ask for net_gas_production by well and you cannot get shrinkage wrong, because nobody is recomputing shrinkage. It was decided upstream, once.
When constrained Text to SQL is fine
Hold does not mean never. It means not the unsupervised, raw-schema version. A constrained setup over a clean semantic layer is a genuinely reasonable thing to build, and the constraints are what make it safe:
- The model queries defined metrics and dimensions, not arbitrary tables. It picks from
net_gas_production,working_interest, and a known dimension list. It cannot invent an allocation. - The generated query is shown to the user before it runs, which is exactly the human review ThoughtWorks requires.[1:3]
- The surface is read-only and scoped. No
UPDATE, noDELETE, no reaching into tables the semantic layer does not expose. - The output range is sanity-checked. A well producing negative gas or ten times its type curve gets flagged, not returned.
Inside those rails, natural language over a semantic layer is closer to a smart metric picker than a code generator, and that is a job an LLM does well. The model is choosing from a menu of correct definitions instead of writing the definitions itself. The hard part, the domain semantics, was already solved by the people who know the difference between gross and net.
That ordering is the whole point. Build the semantic layer first, because you needed it anyway for humans to agree on their numbers. The clean natural-language interface is a feature you can add on top once the definitions are trustworthy. Do it in the other order, model straight onto raw tables, and you have automated the production of confident wrong answers. This is the pattern we keep coming back to on where AI is real and where it is hype: the value shows up when the data foundation is already solid, and evaporates when it is not.
The queue of ad hoc data requests is a real problem worth solving. The semantic layer solves most of it for humans on its own. Whether you then bolt a language model on top is a smaller decision than it looks, and a much safer one once the definitions underneath are correct.