This may well be very useful, but Claude has ruined me and I have a visceral reaction to āgatesā in the opening sentence and couldnāt get any further. If a tool is worth building and sharing, surely itās also worth writing a brief summary of what itās supposed to achieve in your own words?
The sentence is near incomprehensible without knowledge of the Claudish robolect. (Here I made up a word. Hopefully its comprehensibility isn't gated too much on language nerdery.)
Seriously though, it'd be nice if we started using a more shared and less vendor-specific jargon to refer to what these latest features are. We're getting into JVM vs. .NET Execution Engine distinctions and they just muddy the water on what goes on in the market.
It's vague puffery that implies depth without nessesarily achieving it. It's a very effective bullshitting technique with most people, which is why we collectively were so impressed with it we sculpted a robot that does it.
The more common english term is āheld backā instead of gated.
And nvidia probably doesnāt want to say āprobabilistic responses including hallucinations preclude reliable ai analyticsā
As it is the sentence communicates very little. Things are unreliable for a lot of different reasons. Hardware? Software? Users? Intentional and required use of RNGs? Claude presumes the reader has all the same weights activated.
I've been screening some twitch streamers and noticed with some alarm that an unexpected number of seemingly healthy American streamers can't read, or very barely can.
The game I was using for this uses a standard memo pads mechanic to fill in the story and many were barely stumbling through it in their own native language.
So it's my impression that this is not entirely Claude's fault.
Using gate as a verb is extremely rare in American English, though perhaps more common in British English. I would hazard a guess that very few people who aren't exposed to Claude on a regular basis would think that was anything other than gibberish.
As a native speaker, I read this as "This controls enabling or disabling of all analytic features used by AI model loops"
Now there are times when I let the model keep its own docs in its own language, for its own future reference; but, it is not intended for human distribution due to its odd selection of words and phrases (aka slop).
Thank you! After reading the very first line I just had to check the HN comments:
> Agents can retrieve documents and query databases, but reliability gates agentic analytics: a revenue metric answered faster is worth nothing if the number is wrong.
Absolutely no human would ever write this kind of slop. I have no idea what this even means. Is this really the first thing you want users to read?
I wish people would take a fucking minute to at least figure out what their main selling point is and just write it, you know, like we used to.
Agents can retrieve documents and query databases, but retrieval reliability limits the utility of agentic analytic abilities: a revenue metric answered faster is worth nothing if the number is wrong.
i had exactly the same issue. That first sentence triggered the bad experience i had with claude opus 5.0 interactions and i just closed the page immediately.. All of the PRs that I have seen with "gates"... it really impacts me
That's much more like an agent error than a human error.
It's quite common for LLMs to mix up values between columns in a table. It's often referred to as a structural hallucination, since it can happen with all kinds of relationships that form structures, not just columns in a table.
"Ingestion Service" has 3 arrows out and nothing coming in. What is it ingesting? Are DB nodes supposed to show the DB technology they are using or what they store/do? - one node is "PostreSQL" and another node is "Source databases".
Maybe typical marketing power points are worthless. But I called it slop because it didn't help me at all understanding what they are doing.
Yes, it's a professional assessment. This page is 100% unedited slop, and it's unprofessional to publish something like this under the pretense that it's fit for human consumption.
Iāve never shared Claude text output with another human without explicitly flagging it as such. I believe my ability to communicate effectively in my own words is a large part of the value I offer and Claudeās writing (at least in relation to code) is so convoluted and malformed that I wouldnāt dream of attempting to pass it off as my own.
It has a utility in the same way that raw ingredients have a utility in a kitchen. The chef doesnāt just drop a messy pile of uncooked meat and vegetables onto your plate and tell you to figure it out. Thatās exactly what people are doing when they vomit out a bunch of impenetrable slop and expect the reader to do the hard work of comprehending it on their behalf.
You canāt expect others to care about what youāre saying if youāve not spent a moment considering how best to say it. The readerās time is no less valuable than your own. If you think composing a readme for your project is worth zero minutes of your time, thatās the appropriate amount of time for readers to allocate to it.
For whatever itās worth, I also cancelled my Claude subscription recently and shifted back to OpenAI solely because I find its outputs far more readable. Iāll even accept a slight hit on model performance as a trade-off. If Iām unwilling to read Claude-speak when Iām being paid an hourly rate, you can be sure Iām not reading it for free.
> Iāve never shared Claude text output with another human without explicitly flagging it as such.
You've likely shared code written by OpenAI and don't mind subjecting other to it. So why should technical docs be different?
I think you're fighting a losing fight trying to get people to waste their time hand-writing tech docs just to avoid triggering over-sensitive Claude power users because they saw the word "gate".
Code and prose are different things, but that aside, no, I donāt expect anyone I work with to be subject to my mindless one-shotted PRs ā which is as close a comparison I can think of on the development side. I also wouldnāt consider a readme to be a pure technical doc. Itās very likely to be the first, and possibly last, impression someone forms of your project. It matters.
Iām not fighting any particular fight, Iām just saying that the lack of care stands out to me, and evidently to others, and that anyone releasing a new project might want to consider whether they want it to be perceived in that way.
> Iām just saying that the lack of care stands out to me
You don't know how much they cared. You also don't know if they reviewed the README before publishing, or if was a mindless one-shotted PR that nobody looked at. The word "gate" isn't a smoking gun here, other than to prove that the reader uses AI a ton and is ironically now triggered by it. If your criticism was more substantial: targeting the content rather than the use of a trigger word, then I might agree. But you haven't done anything but project your own bias.
āGateā was just the trigger to quickly scan the rest of the copy, at which point AIās fingerprints are so obvious that itās impossible to believe a human wrote it. I havenāt bothered to check, but Iām sure any of the AI detector sites would light up like a Christmas tree on this.
These criticisms are empty, though. Just because it's AI-generated doesn't make it bad. You're not saying what's bad about it other than "I can tell AI wrote it, therefore I reject it". Which imo is wildly hypocritical considering you power use the exact same AI tools to auto-generate output.
it's full of AI Slop fingerprints. Clearly hasn't been read or edited by a human before being published, or they're already deep in the brain rot that they can't see just how bad it is.
Great, as long as they never share any of their Claude output (to include code) with anyone.
But even better: if Claude's output is so torturous, they should probably just stop using it altogether rather than thumbing their nose at other people that use it.
I want to attempt an explanation as it seems a lot of folks here didn't grok the very terribly written README
"Auto Ontology" is sort of a poor name.
Here's the problem:
Your company has data scattered across multiple types of databases. People who make decisions want to ask questions about that data.
The old way is they would go to a Data Analyst, and say, "Get me a report on so and so", and it was the Analyst's job to iron out the right series of queries and one-off scripts to generate the needed info.
With a system like this, the idea is that all your database models have been introspected and stored in a central metadata catalogue.
When a question is asked like "What were the top selling items in the month of May?", the metadata catalogue tells you "Products are stored in FOO_DB" and "Product_Sales are stored in BAR DB" and so you go and fetch the right data from the right place and do whatever in-memory calculations you need to do.
What the system provides is the implementation code for:
- Introspecting each of your DB's
- Aggregating model + metadata information in a central, virtual DB (VDB)
- Generating dialect-specific SQL from natural language queries and routing them to the right DB for the underlying table
- Combining results in-memory when a natural language query requires multiple datasources
- Handling proper permissions, etc
A GOOD system for doing this also integrates with your external API sources (OpenAPI, GraphQL, gRPC, whatever) and also documentation sources (cough, I work on this, cough).
what happens when your column names dont accurately represent what data is being stored (i know i wish it did but it doesnt at my place right now) and its all esoteric knowledge held by our data scientists? my worry is now the guys above will think its answering their question without knowing that its either A. wrong or B. data they may not be able to use for legal reasons on the project they are trying to make.
"seems a lot of folks here didn't grok the very terribly written README"
I tried to make it through the README and it was just unmotivated slop. Perhaps it reads better to agents. It is like the Van Halen rider story - if the authors didn't care to write a good README, what are the odds that they paid attention to the rest of the large codebase?
I ended up throwing the README at an LLM and it gave me a much simpler and clearer explanation. Copying the bits of an explanation that helped me:
> The problem itās trying to solve is that generating valid SQL doesnāt necessarily produce the right answer. An AI might find a column called revenue, but still choose the wrong dataset or misunderstand whether that figure includes things like refunds. Auto Ontology adds information about what the data means to help it make those decisions.
and then the one paragraph summary at the end:
> My reading: itās a database chat application built around a reusable catalogue of business meaning. The question worth testing is whether that catalogue makes answers measurably more correctāand how much human work it takes to get the definitions right.
Fixing unreliable agents by adding more unreliable agents. It's a shame we can't do something about the tendency of LLMs to give more weight to their weights than to the data they're analysing.
I guess it all comes back to the fact that they don't have a clear division between data they have learned about the world, instructions and data under analysis.
You can ask them only to consider the 5 facts in front of them, but they won't do that, and cannot do that.
> Fixing unreliable agents by adding more unreliable agents.
Depending on how you combine imperfect systems, you can lower or increase error rates.
> data they have learned about the world, instructions and data under analysis.
I don't think there really is a clear distinction between these things though.
And I don't mean this in a gotcha way - these things apply to people too. Not that that means they're like people, but we have successfully built enormous systems out of fallible people, and other imperfect things.
The goal is rarely "eliminate all errors" but "have a thing that's useful, given a possibility of things going wrong".
> I don't think there really is a clear distinction between these things though.
Computer systems, though, can have that distinction. If you read data supplied by the user, and then process it via behavior in code, and process that result against administratively-set data in the database, the separation between user-supplied data, administratively-configured data, and instructions (code) is extremely clear. Even if you are doing something that could be nondeterministic, e.g. incorporating randomness or doing sentiment analysis or whatnot.
I don't think the oft-repeated "AI is fallible just like humans are" applies here. Computer systems are an area where that separation can be achieved. We know this because we've achieved it a lot. LLMs, powerful as they are, are a massive step back in that regard. Their benefits might outweigh that compromise, but it is, as they say, "worth naming" anyway.
> Computer systems, though, can have that distinction. If you read data supplied by the user, and then process it via behavior in code, and process that result against administratively-set data in the database, the separation between user-supplied data, administratively-configured data, and instructions (code) is extremely clear
No it is not.
They are three places you can put things, amongst an infinite other number of places you can, but they are not some inherent core distinctions.
Is a javascript file code or data? If I read it to measure file size and don't execute it, is it ever code? If I read in a file and do different things in the code based on it, and never alter that file, how is that any different from having if (x) baked into the code? Why is it user supplied data if we read in a file and do things based on the contents yet code when I read in js and execute stuff in a VM?
They are just different places with different access controls that alter the outcome. There's no magical distinction between code, data and admin data or anything else other than really who gets to change what. And then there's obviously not just three named buckets of "stuff", it's as many as you want.
> I don't think the oft-repeated "AI is fallible just like humans are" applies here.
You're missing the point here, the point is we have built big things with fallible components. Therefore the idea that a component is fallible does not just from that fact mean we can't do useful things with it. And it's a very relevant point if we're looking at "where can we use this in place of humans" - if we had a human there before we were not starting from a perfectly reliable point.
Depending on how you combine imperfect systems, you can lower or increase error rates.
Well, in principle yes one system can compensate for another, in practice, errors tend to compound and combine in interesting ways, not go away, particularly in systems like this which are unpredictable and generate slightly different responses each time without human intervention. They are not reliable in the way we're used to considering and don't work well without supervision from careful humans at present.
I don't think there really is a clear distinction between these things though.
Yes your second point is an interesting one - the divisions between these things are arbitrarily imposed on the world, but we do have these distinctions for a reason, even if they are not completely objective.
I do think this is a significant problem specific to LLMs though (that I at least see when using them) - we don't have a way to constrain them in what knowledge they consider. This can be a bonus (they are good at connecting faraway bits of maths in interesting ways), but it can be a significant problem when asking for a summary of a data - confabulations and data from outside the context easily leak into the output of the LLM. That is undesirable behaviour and we have no reliable way to constrain it.
I'm not sure that this particular effort is going to help because it doesn't change or constrain the behaviour of the actual machines and the behaviour is the problem, not the data they are fed.
> Well, in principle yes one system can compensate for another, in practice, errors tend to compound and combine in interesting ways, not go away, particularly in systems like this which are unpredictable and generate slightly different responses each time without human intervention.
Ah but the point here IIUC is moving from "let an LLM figure out the right way of querying the data and what you mean with your terms from a wide span of different documents and sources, which it has to perform multiple searches on" to "let an LLM figure out the right way of querying your data with a coherent and machine readable ontology".
While the conversion from "load of data" to "defined structure" is error prone, you now have the second step operating on the same converted thing, and you have ways of fixing that over time (the corrections).
By splitting the task into two distinct phases, you can both interpret and fix the middle step, and remove a large class of mistakes and/or differences caused by different interpretations of the same documents.
There are really two answers to it IMO that this boils down to:
1. This process may be less error prone for that first step than the LLM you're using. That could reduce errors.
2. This process means the first step is run once rather than every time which means you cut out one area where there can be differences each time.
> Yes your second point is an interesting one - the divisions between these things are arbitrarily imposed on the world, but we do have these distinctions for a reason, even if they are not completely objective.
You're right, I think there are useful groupings, but I think any effort to fully isolate them is doomed to fail because they inherently blend. Your instructions contain things that can only be understood in context of world knowledge and the data. The data only makes sense with certain bits of world knowledge.
We also know that these things happen with humans (again just to pre-empt how others may read this and not saying you are accusing me of this, I'm not gotcha-ing here) and so we often have to be careful about how we ask questions, how different cultural backgrounds impact understanding and so forth. We have to take care about how we structure things.
> I'm not sure that this particular effort is going to help because it doesn't change or constrain the behaviour of the actual machines and the behaviour is the problem, not the data they are fed.
I think this effort does address that. A really key thing is that this is about turning human readable questions into SQL that then only query the actual data and you see the result of that query - it allows you to constrain things more (at both points).
Yes, also I hate the direction "frontier" is taking now. Opus 5.5 may be better in agentic coding and reverse engineering, but it has regressed greatly in instruction following.
You can give it instructions and 5k tokens later it will do exact opposite you told it to.
> Fixing unreliable agents by adding more unreliable agents.
I think the 'reliability add' is from how it's presented back to the user. If you can trust the agent to at least say what tool it called, then you can mark a chart as being the canonical output of "get_projected_profit" tool vs "get_realised_profit" tool and everyone knows it hasn't come down to the current dice roll of randomness what the agent itself did this time.
Reliable TCP was built on top of unreliable IP. Our body chemistry is another example of unreliable non-deterministic processes creating more deterministic outcomes. Yes, yes, it's all different. The common idea is the same: we can get reliable results out of unreliable systems.
I don't know enough about it, would welcome a more informed opinion on that, as a user it's the biggest issue with using them long-term IMO. Perhaps it is fixable somehow, and perhaps 'cannot do that' is a bit strong, what I meant was the current iteration of LLMs cannot do that.
Surprised that https://github.com/ucbepic/DataAgentBench hasn't been mentioned on this: benchmarking data agents that need to query from multiple sources is a focus of some research at UC Berkeley -- leaderboard is worth a look to see how the various closed and open-source implementations build ontologies ... would like to see how Nvidia's approach stacks up.
Iām struggling to see why this is better than OWL/RDF and SparQL. Given the breadth and depth of semantic web knowledge in the corpus they do an amazing job discovering well established ontologies and building new ones on presented data and querying it with sparql. In fact better than almost anything Iāve seen. What an irony that the semantic web after all these years is likely to find a real use - itās original one in fact.
I am new to this field and our org struggle with authoring and documenting existing implicitly given ontologies. It is like working with archeology, and proposed automated solutions like nvidiaās seem alluring in that regard. What would be your suggestion for a way to extract the ontology/RDF/OWL/graph from existing underdocumented fields?
it's local first, agent-friendly tooling; you have single cli to access it (mcp is there as well if that's your thing)
everything is persisted in simple local text files that you can actually read (or sqlite database if you don't care / have more data)
ie there is no integration, k8s-style nonsense etc, you can just use it on project or on your system user level and know exactly what's there, you have control over it, you can delete it, move it, extract parts you find useful and move on etc
because it has first class support for uncertainty
evidence is first class concern
supports evolution of belief (statements can be revised, retracted etc. without destroying history)
knowledge is executable (rules, actions, automations ā all operate on the same claim model)
it works well for knowledge that keeps changing (including support for querying knowledge as of some past time)
it's great for llm sessions, ie. simple things like comment on a claim being part of a claim itself, which means it appears when working with knowledge, which allows to support weird/human/natural-language aspects that are not directly part of the model
...
rdf/owl/sparql is great for large, public, structured datasets (ie. medical, scientific or some regulatory, standardized ontologies)
cave is great for evolving knowledge, ie. in most cases i use it by just dropping it in and forgetting about it ā i don't care how/what llm will model, i just care it'll solve some problems faster when aiding itself with it as a tool; if i want some structured dataset out of it i may occasionally ask it to model it around homogeneous schema i can extract, but in most cases i really don't care
I have a question for people with knowledge of both LLMs and GOFAI: Why canāt we use LLMs to write as many symbolic rules as needed to fit the data while keeping the ruleset reviewable and maintainable?
The amount of finite rules necessary (that the LLM will write) to handle the noise and complexity of real data generates such a dense graph that it breaks the premise of a human-maintainable and auditable system.
sorry this may be way off, but feels analogous to me.
reminds me of http://www.thai-language.com/ (it's safe). it was around long before Google Translate got good enough. i don't know the authors/maintainers, but afaik, everything is entered manually. this is the "analytical" approach to language. most translations before Google Translate went full ML used an analytical approach.
anyway, long story short, turns out "analyzing" a language to "understand" it GREATLY underperforms probabilistic/statistical methods like ML.
also reminds me of trying to label parts of speech in Vietnamese. there's a school of thought that finds it useful...then there's another school of thought that plainly points out words that can be verbs, nouns, adjectives all in different contexts.
Shameless plug: we built our own open source equivalent of auto-ontology a few months ago, that doesn't require any API key but your LLM one (or none if you use your coding agent subscription)
We spend a small fortune trying to get ontology right as it's so critical to real world success of enterprise AI. Curious if this could actually help us.
Nvidia, I get you're all-in on AI, but please at least find a human to write some copy. It's both overly verbose and entirely lacking of important info at the same time.
I don't understand this ontology thing. What's the difference between this and a memory system plus skills to collect your enterprise's APIs and CLIs etc?
This is more about streamlining the Date Engineer / Data Scientist role. Business Intelligence, in other words.
Before: domain expert wants to ask a question about the data, but which of these hundred tables actually has the right ārevenue per Userā column? Ask a DE to help slice the data and bless the queries.
After: ask your agent. The agent looks at the ontology and can usually find the best blessed data. To enable this you need to write down all the tribal knowledge that lived in the DE/DS head; agents can take the first pass at proposing and updating metadata.
Under the covers the DE and DS roles switch to reviewing the underlying ontology, instead of working directly with business stakeholders on one-off queries.
Ontology is a so-called white-box approach for AI: all dependencies are clearly stated and formalised. If something does not work as you wish, unlike LLMs, in ontology world you can clearly diagnose where the system fails. I understood the NVIDIA's idea this way: we take unreliable LLM and make a "reliable" ontology using it.
If you look back into history, Prolog , was exactly a sort of "memory system" or, better put, a human-friendly database. Ontology is a further development of this concept.
1. What is not in the codebase. Obvious of course, but building an ontology and maintaining it over time forces a company to document what was only living in the head of some experts and never written down. In the case of analytics use cases, think of all the quirks a data analyst might have in mind when querying the data. Very basic e.g. "exclude test accounts whenever you count customers". Every analyst on the team does it without thinking, it's rarely written anywhere, and an agent that doesn't know will happily count them.
2. Related to 1., your codebase or source material may hold different interpretations of the same definition (what "revenue" means can differ between Sales and Finance). The ontology forces you to choose one, or to state when to use each. Without that, the LLM may pick one a bit randomly.
3. It's a governance tool for humans. It gives the company a human-readable, "deterministic" source of truth to agree on what means what, then version it and evolve it.
One issue in a business is what does all the data actually mean? What are the various definitions and how do they relate to each other. What's a customer? Should that include people that have bought things, or anyone who is potentially buying? Do different systems use the same definitions, what are the columns, etc.
An ontology in this kind of context means a more precise machine readable spec of this and how it all relates to each other. You can totally build one by hand. The idea here is I think that it tries to figure it out itself and can take corrections over time, so you can ask natural language questions and get more reliable results rather than hoping an llm works out what "customers this month" means the same way each time over a wide span of data rather than having a much clearer API to work against.
This is what the Matt Pocock /grill-with-docs solves. It helps to build a ubiquitous language that helps with domain modelling. This along with the codebase seems to help agents a lot. I'm not clear on how the Nvidia tool here improves on this, or how it is more reliable?
A memory system is managed by the RL trained behavior, these systems use specific prompts and ordinary software to hardcode a policy. The RL version is usually more efficient, but it can't be improved or made more repeatable.
How about we start by open-sourcing the NVIDIA user-space libraries: libcuda, libcudart, libOpenCL, libnvtx, as well as nvidia-smi and the CUDA and OpenCL profilers.
Nice is this Jensen Huang response's to Alex Karp recent statement on AI companies stealing people data with a subvention (token price) ? Because building ontologies is Palantir's core business.
The readme seems to be written for enterprise project managers. Does anyone know what philosophy or academic ontology work this ties to, if any? Iām having trouble understanding the specifics, Iāll be honestā¦
This isn't really anything to do with ontology in the philosophical or academic sense, outside of computer science.
In software, an ontology is just a machine-readable definition of the entities in a system - their names, properties, relationships etc. This tool analyzes a system and produces such a definition for agents to use.
This may well be very useful, but Claude has ruined me and I have a visceral reaction to āgatesā in the opening sentence and couldnāt get any further. If a tool is worth building and sharing, surely itās also worth writing a brief summary of what itās supposed to achieve in your own words?
The sentence is near incomprehensible without knowledge of the Claudish robolect. (Here I made up a word. Hopefully its comprehensibility isn't gated too much on language nerdery.)
I will add Claudish robolect into my language immediately, thank you.
Hear ye, who speaks of the Pompatus of Claudeā¦
Claudish is a perfectly cromulent word.
Seriously though, it'd be nice if we started using a more shared and less vendor-specific jargon to refer to what these latest features are. We're getting into JVM vs. .NET Execution Engine distinctions and they just muddy the water on what goes on in the market.
It's particularly load-bearing to a non-native English speaker such as me.
It looks a lot like English, but it doesn't have the same rhythm human English has. It's as if a human tried and failed to be concise.
As a native English speaker, I get the same feeling whenever I read press releases and other announcements from executives.
It's vague puffery that implies depth without nessesarily achieving it. It's a very effective bullshitting technique with most people, which is why we collectively were so impressed with it we sculpted a robot that does it.
Dailect
"reliability gates agentic analytics" is quite a garden path sentence, at least to the eyes of a non-native speaker.
The more common english term is āheld backā instead of gated.
And nvidia probably doesnāt want to say āprobabilistic responses including hallucinations preclude reliable ai analyticsā
As it is the sentence communicates very little. Things are unreliable for a lot of different reasons. Hardware? Software? Users? Intentional and required use of RNGs? Claude presumes the reader has all the same weights activated.
I've used the word "Gate" in this context.
People are really seeing 'AI' in everything now. You can't tell.
Humans are now hallucinating what is, or is not, AI.
I've been screening some twitch streamers and noticed with some alarm that an unexpected number of seemingly healthy American streamers can't read, or very barely can.
The game I was using for this uses a standard memo pads mechanic to fill in the story and many were barely stumbling through it in their own native language.
So it's my impression that this is not entirely Claude's fault.
Conservatively, I'll blame it on "dyslexia".
Using gate as a verb is extremely rare in American English, though perhaps more common in British English. I would hazard a guess that very few people who aren't exposed to Claude on a regular basis would think that was anything other than gibberish.
LOL
People are "gatekeeping" how we use the word "gate".
So you're claiming a Claude-shaped readability gate? /s
As a native speaker, I read this as "This controls enabling or disabling of all analytic features used by AI model loops"
Now there are times when I let the model keep its own docs in its own language, for its own future reference; but, it is not intended for human distribution due to its odd selection of words and phrases (aka slop).
I think it's slightly different: Without reliability, agentic analytics is useless.
Or: Agentic analytics is gated on sufficient reliabilty.
Or reallllly short and hard to understand: Reliability gates agentic analytics.
I read this whole readme and I have no freaking clue what this does beyond "searchable knowledgebase". How about providing some use cases?
Thank you! After reading the very first line I just had to check the HN comments:
> Agents can retrieve documents and query databases, but reliability gates agentic analytics: a revenue metric answered faster is worth nothing if the number is wrong.
Absolutely no human would ever write this kind of slop. I have no idea what this even means. Is this really the first thing you want users to read?
I wish people would take a fucking minute to at least figure out what their main selling point is and just write it, you know, like we used to.
Agents can retrieve documents and query databases, but retrieval reliability limits the utility of agentic analytic abilities: a revenue metric answered faster is worth nothing if the number is wrong.
Just adding a few words takes it so far...
i had exactly the same issue. That first sentence triggered the bad experience i had with claude opus 5.0 interactions and i just closed the page immediately.. All of the PRs that I have seen with "gates"... it really impacts me
Opus 5.0 lingo left a lasting psychological damage on developers' psyches. And some good memes related to it.
yes pls linguists track these word overuses and help us get rid. Everytime someone says Gate IRL i lose my train of thaught
And yet it still contains what looks like human error, e.g. the "Software Components" table mixed up the columns for the database.
That's much more like an agent error than a human error.
It's quite common for LLMs to mix up values between columns in a table. It's often referred to as a structural hallucination, since it can happen with all kinds of relationships that form structures, not just columns in a table.
Hmm, seems their ontology didn't fix this assuming they use it themselves.
Is that first diagram also pure slop? I would have to understand more of what they're doing to confidently say it is, but it seems odd.
All the four nodes on the right side have either only incoming or only outgoing arrows.
Typical marketing blocks in a power point. Not everything is AI slop, anymore than marketing has been 'slop'.
What? now I can't draw boxes and arrows without it being slop?
"Ingestion Service" has 3 arrows out and nothing coming in. What is it ingesting? Are DB nodes supposed to show the DB technology they are using or what they store/do? - one node is "PostreSQL" and another node is "Source databases".
Maybe typical marketing power points are worthless. But I called it slop because it didn't help me at all understanding what they are doing.
Seriously? This is a professional assessment?
> If a tool is worth building and sharing, surely itās also worth writing a brief summary of what itās supposed to achieve in your own words?
Next the complaint will be any such description looks like AI slop.
Yes, it's a professional assessment. This page is 100% unedited slop, and it's unprofessional to publish something like this under the pretense that it's fit for human consumption.
We cant use single English words in their normal sense now without being told it's Claude?
Read the rest of the page. This is 100% unedited slop.
More like, sometimes when you publish slop it gets called out as slop.
Thanks Claude. /s
> Claude has ruined me and I have a visceral reaction to āgatesā in the opening sentence and couldnāt get any further
So what you're saying is: because you've used Claude so much, you don't want other people to use it. And you reject them for using it.
A bit hypocritical don't you think?
There's a difference between using Claude and sharing its unfiltered output.
Iāve never shared Claude text output with another human without explicitly flagging it as such. I believe my ability to communicate effectively in my own words is a large part of the value I offer and Claudeās writing (at least in relation to code) is so convoluted and malformed that I wouldnāt dream of attempting to pass it off as my own.
It has a utility in the same way that raw ingredients have a utility in a kitchen. The chef doesnāt just drop a messy pile of uncooked meat and vegetables onto your plate and tell you to figure it out. Thatās exactly what people are doing when they vomit out a bunch of impenetrable slop and expect the reader to do the hard work of comprehending it on their behalf.
You canāt expect others to care about what youāre saying if youāve not spent a moment considering how best to say it. The readerās time is no less valuable than your own. If you think composing a readme for your project is worth zero minutes of your time, thatās the appropriate amount of time for readers to allocate to it.
For whatever itās worth, I also cancelled my Claude subscription recently and shifted back to OpenAI solely because I find its outputs far more readable. Iāll even accept a slight hit on model performance as a trade-off. If Iām unwilling to read Claude-speak when Iām being paid an hourly rate, you can be sure Iām not reading it for free.
> Iāve never shared Claude text output with another human without explicitly flagging it as such.
You've likely shared code written by OpenAI and don't mind subjecting other to it. So why should technical docs be different?
I think you're fighting a losing fight trying to get people to waste their time hand-writing tech docs just to avoid triggering over-sensitive Claude power users because they saw the word "gate".
Code and prose are different things, but that aside, no, I donāt expect anyone I work with to be subject to my mindless one-shotted PRs ā which is as close a comparison I can think of on the development side. I also wouldnāt consider a readme to be a pure technical doc. Itās very likely to be the first, and possibly last, impression someone forms of your project. It matters.
Iām not fighting any particular fight, Iām just saying that the lack of care stands out to me, and evidently to others, and that anyone releasing a new project might want to consider whether they want it to be perceived in that way.
> Iām just saying that the lack of care stands out to me
You don't know how much they cared. You also don't know if they reviewed the README before publishing, or if was a mindless one-shotted PR that nobody looked at. The word "gate" isn't a smoking gun here, other than to prove that the reader uses AI a ton and is ironically now triggered by it. If your criticism was more substantial: targeting the content rather than the use of a trigger word, then I might agree. But you haven't done anything but project your own bias.
āGateā was just the trigger to quickly scan the rest of the copy, at which point AIās fingerprints are so obvious that itās impossible to believe a human wrote it. I havenāt bothered to check, but Iām sure any of the AI detector sites would light up like a Christmas tree on this.
These criticisms are empty, though. Just because it's AI-generated doesn't make it bad. You're not saying what's bad about it other than "I can tell AI wrote it, therefore I reject it". Which imo is wildly hypocritical considering you power use the exact same AI tools to auto-generate output.
it's full of AI Slop fingerprints. Clearly hasn't been read or edited by a human before being published, or they're already deep in the brain rot that they can't see just how bad it is.
What they're saying is that if something is not worth the time needed to write it, it's not worth the time needed to read it either.
Great, as long as they never share any of their Claude output (to include code) with anyone.
But even better: if Claude's output is so torturous, they should probably just stop using it altogether rather than thumbing their nose at other people that use it.
I want to attempt an explanation as it seems a lot of folks here didn't grok the very terribly written README
"Auto Ontology" is sort of a poor name.
Here's the problem:
Your company has data scattered across multiple types of databases. People who make decisions want to ask questions about that data.
The old way is they would go to a Data Analyst, and say, "Get me a report on so and so", and it was the Analyst's job to iron out the right series of queries and one-off scripts to generate the needed info.
With a system like this, the idea is that all your database models have been introspected and stored in a central metadata catalogue.
When a question is asked like "What were the top selling items in the month of May?", the metadata catalogue tells you "Products are stored in FOO_DB" and "Product_Sales are stored in BAR DB" and so you go and fetch the right data from the right place and do whatever in-memory calculations you need to do.
What the system provides is the implementation code for:
- Introspecting each of your DB's
- Aggregating model + metadata information in a central, virtual DB (VDB)
- Generating dialect-specific SQL from natural language queries and routing them to the right DB for the underlying table
- Combining results in-memory when a natural language query requires multiple datasources
- Handling proper permissions, etc
A GOOD system for doing this also integrates with your external API sources (OpenAPI, GraphQL, gRPC, whatever) and also documentation sources (cough, I work on this, cough).
what happens when your column names dont accurately represent what data is being stored (i know i wish it did but it doesnt at my place right now) and its all esoteric knowledge held by our data scientists? my worry is now the guys above will think its answering their question without knowing that its either A. wrong or B. data they may not be able to use for legal reasons on the project they are trying to make.
"seems a lot of folks here didn't grok the very terribly written README"
I tried to make it through the README and it was just unmotivated slop. Perhaps it reads better to agents. It is like the Van Halen rider story - if the authors didn't care to write a good README, what are the odds that they paid attention to the rest of the large codebase?
I ended up throwing the README at an LLM and it gave me a much simpler and clearer explanation. Copying the bits of an explanation that helped me:
> The problem itās trying to solve is that generating valid SQL doesnāt necessarily produce the right answer. An AI might find a column called revenue, but still choose the wrong dataset or misunderstand whether that figure includes things like refunds. Auto Ontology adds information about what the data means to help it make those decisions.
and then the one paragraph summary at the end:
> My reading: itās a database chat application built around a reusable catalogue of business meaning. The question worth testing is whether that catalogue makes answers measurably more correctāand how much human work it takes to get the definitions right.
I had to go digging into the source to realize what it was, e.g.
https://github.com/NVIDIA/auto-ontology/blob/main/auto_ontol...
Fixing unreliable agents by adding more unreliable agents. It's a shame we can't do something about the tendency of LLMs to give more weight to their weights than to the data they're analysing.
I guess it all comes back to the fact that they don't have a clear division between data they have learned about the world, instructions and data under analysis.
You can ask them only to consider the 5 facts in front of them, but they won't do that, and cannot do that.
> Fixing unreliable agents by adding more unreliable agents.
Depending on how you combine imperfect systems, you can lower or increase error rates.
> data they have learned about the world, instructions and data under analysis.
I don't think there really is a clear distinction between these things though.
And I don't mean this in a gotcha way - these things apply to people too. Not that that means they're like people, but we have successfully built enormous systems out of fallible people, and other imperfect things.
The goal is rarely "eliminate all errors" but "have a thing that's useful, given a possibility of things going wrong".
> I don't think there really is a clear distinction between these things though.
Computer systems, though, can have that distinction. If you read data supplied by the user, and then process it via behavior in code, and process that result against administratively-set data in the database, the separation between user-supplied data, administratively-configured data, and instructions (code) is extremely clear. Even if you are doing something that could be nondeterministic, e.g. incorporating randomness or doing sentiment analysis or whatnot.
I don't think the oft-repeated "AI is fallible just like humans are" applies here. Computer systems are an area where that separation can be achieved. We know this because we've achieved it a lot. LLMs, powerful as they are, are a massive step back in that regard. Their benefits might outweigh that compromise, but it is, as they say, "worth naming" anyway.
> Computer systems, though, can have that distinction. If you read data supplied by the user, and then process it via behavior in code, and process that result against administratively-set data in the database, the separation between user-supplied data, administratively-configured data, and instructions (code) is extremely clear
No it is not.
They are three places you can put things, amongst an infinite other number of places you can, but they are not some inherent core distinctions.
Is a javascript file code or data? If I read it to measure file size and don't execute it, is it ever code? If I read in a file and do different things in the code based on it, and never alter that file, how is that any different from having if (x) baked into the code? Why is it user supplied data if we read in a file and do things based on the contents yet code when I read in js and execute stuff in a VM?
They are just different places with different access controls that alter the outcome. There's no magical distinction between code, data and admin data or anything else other than really who gets to change what. And then there's obviously not just three named buckets of "stuff", it's as many as you want.
> I don't think the oft-repeated "AI is fallible just like humans are" applies here.
You're missing the point here, the point is we have built big things with fallible components. Therefore the idea that a component is fallible does not just from that fact mean we can't do useful things with it. And it's a very relevant point if we're looking at "where can we use this in place of humans" - if we had a human there before we were not starting from a perfectly reliable point.
Depending on how you combine imperfect systems, you can lower or increase error rates.
Well, in principle yes one system can compensate for another, in practice, errors tend to compound and combine in interesting ways, not go away, particularly in systems like this which are unpredictable and generate slightly different responses each time without human intervention. They are not reliable in the way we're used to considering and don't work well without supervision from careful humans at present.
I don't think there really is a clear distinction between these things though.
Yes your second point is an interesting one - the divisions between these things are arbitrarily imposed on the world, but we do have these distinctions for a reason, even if they are not completely objective.
I do think this is a significant problem specific to LLMs though (that I at least see when using them) - we don't have a way to constrain them in what knowledge they consider. This can be a bonus (they are good at connecting faraway bits of maths in interesting ways), but it can be a significant problem when asking for a summary of a data - confabulations and data from outside the context easily leak into the output of the LLM. That is undesirable behaviour and we have no reliable way to constrain it.
I'm not sure that this particular effort is going to help because it doesn't change or constrain the behaviour of the actual machines and the behaviour is the problem, not the data they are fed.
> Well, in principle yes one system can compensate for another, in practice, errors tend to compound and combine in interesting ways, not go away, particularly in systems like this which are unpredictable and generate slightly different responses each time without human intervention.
Ah but the point here IIUC is moving from "let an LLM figure out the right way of querying the data and what you mean with your terms from a wide span of different documents and sources, which it has to perform multiple searches on" to "let an LLM figure out the right way of querying your data with a coherent and machine readable ontology".
While the conversion from "load of data" to "defined structure" is error prone, you now have the second step operating on the same converted thing, and you have ways of fixing that over time (the corrections).
By splitting the task into two distinct phases, you can both interpret and fix the middle step, and remove a large class of mistakes and/or differences caused by different interpretations of the same documents.
There are really two answers to it IMO that this boils down to:
1. This process may be less error prone for that first step than the LLM you're using. That could reduce errors. 2. This process means the first step is run once rather than every time which means you cut out one area where there can be differences each time.
> Yes your second point is an interesting one - the divisions between these things are arbitrarily imposed on the world, but we do have these distinctions for a reason, even if they are not completely objective.
You're right, I think there are useful groupings, but I think any effort to fully isolate them is doomed to fail because they inherently blend. Your instructions contain things that can only be understood in context of world knowledge and the data. The data only makes sense with certain bits of world knowledge.
We also know that these things happen with humans (again just to pre-empt how others may read this and not saying you are accusing me of this, I'm not gotcha-ing here) and so we often have to be careful about how we ask questions, how different cultural backgrounds impact understanding and so forth. We have to take care about how we structure things.
> I'm not sure that this particular effort is going to help because it doesn't change or constrain the behaviour of the actual machines and the behaviour is the problem, not the data they are fed.
I think this effort does address that. A really key thing is that this is about turning human readable questions into SQL that then only query the actual data and you see the result of that query - it allows you to constrain things more (at both points).
Yes, also I hate the direction "frontier" is taking now. Opus 5.5 may be better in agentic coding and reverse engineering, but it has regressed greatly in instruction following.
You can give it instructions and 5k tokens later it will do exact opposite you told it to.
> Fixing unreliable agents by adding more unreliable agents.
I think the 'reliability add' is from how it's presented back to the user. If you can trust the agent to at least say what tool it called, then you can mark a chart as being the canonical output of "get_projected_profit" tool vs "get_realised_profit" tool and everyone knows it hasn't come down to the current dice roll of randomness what the agent itself did this time.
They are attempting to fix an inherent problem with the technology via epicycles.
Reliable TCP was built on top of unreliable IP. Our body chemistry is another example of unreliable non-deterministic processes creating more deterministic outcomes. Yes, yes, it's all different. The common idea is the same: we can get reliable results out of unreliable systems.
> You can ask them only to consider the 5 facts in front of them, but they won't do that, and cannot do that.
Has training them to do it been tried and proven to fail?
I don't know enough about it, would welcome a more informed opinion on that, as a user it's the biggest issue with using them long-term IMO. Perhaps it is fixable somehow, and perhaps 'cannot do that' is a bit strong, what I meant was the current iteration of LLMs cannot do that.
Surprised that https://github.com/ucbepic/DataAgentBench hasn't been mentioned on this: benchmarking data agents that need to query from multiple sources is a focus of some research at UC Berkeley -- leaderboard is worth a look to see how the various closed and open-source implementations build ontologies ... would like to see how Nvidia's approach stacks up.
Iām struggling to see why this is better than OWL/RDF and SparQL. Given the breadth and depth of semantic web knowledge in the corpus they do an amazing job discovering well established ontologies and building new ones on presented data and querying it with sparql. In fact better than almost anything Iāve seen. What an irony that the semantic web after all these years is likely to find a real use - itās original one in fact.
I am new to this field and our org struggle with authoring and documenting existing implicitly given ontologies. It is like working with archeology, and proposed automated solutions like nvidiaās seem alluring in that regard. What would be your suggestion for a way to extract the ontology/RDF/OWL/graph from existing underdocumented fields?
Few thoughts:
- you need some engine to support your owl/rdf/sparql, and all available are lower quality/perf/features than pg
- its easier for LLMs to talk standard SQL compared so some version of sparql
Arenāt these data formats and query engines? You still need something to extract and maintain the ontology no?
I still prefer my own, simple and more functional CAVE lang [0]
[0] https://mirekrusin.com/cave
Is this similar to Datalog where you have facts and rules?
Yes, and few other useful concepts beyond just Datalog.
why would i use this over some rdf / owl / sparql thing?
i want to use these semantic / KG tech but i find them so confusing!
This seem similar to what using semantic tech (and quad stores) can be used accomplish (Subject, Predicate, Object, Graph).
But you are putting broad but is more opinionated design structure around it. I will have a look thanks.
because it doesn't suck :)
it's local first, agent-friendly tooling; you have single cli to access it (mcp is there as well if that's your thing)
everything is persisted in simple local text files that you can actually read (or sqlite database if you don't care / have more data)
ie there is no integration, k8s-style nonsense etc, you can just use it on project or on your system user level and know exactly what's there, you have control over it, you can delete it, move it, extract parts you find useful and move on etc
because it has first class support for uncertainty
evidence is first class concern
supports evolution of belief (statements can be revised, retracted etc. without destroying history)
knowledge is executable (rules, actions, automations ā all operate on the same claim model)
it works well for knowledge that keeps changing (including support for querying knowledge as of some past time)
it's great for llm sessions, ie. simple things like comment on a claim being part of a claim itself, which means it appears when working with knowledge, which allows to support weird/human/natural-language aspects that are not directly part of the model
...
rdf/owl/sparql is great for large, public, structured datasets (ie. medical, scientific or some regulatory, standardized ontologies)
cave is great for evolving knowledge, ie. in most cases i use it by just dropping it in and forgetting about it ā i don't care how/what llm will model, i just care it'll solve some problems faster when aiding itself with it as a tool; if i want some structured dataset out of it i may occasionally ask it to model it around homogeneous schema i can extract, but in most cases i really don't care
I have a question for people with knowledge of both LLMs and GOFAI: Why canāt we use LLMs to write as many symbolic rules as needed to fit the data while keeping the ruleset reviewable and maintainable?
The amount of finite rules necessary (that the LLM will write) to handle the noise and complexity of real data generates such a dense graph that it breaks the premise of a human-maintainable and auditable system.
sorry this may be way off, but feels analogous to me.
reminds me of http://www.thai-language.com/ (it's safe). it was around long before Google Translate got good enough. i don't know the authors/maintainers, but afaik, everything is entered manually. this is the "analytical" approach to language. most translations before Google Translate went full ML used an analytical approach.
anyway, long story short, turns out "analyzing" a language to "understand" it GREATLY underperforms probabilistic/statistical methods like ML.
also reminds me of trying to label parts of speech in Vietnamese. there's a school of thought that finds it useful...then there's another school of thought that plainly points out words that can be verbs, nouns, adjectives all in different contexts.
We can and do! Here's one you can help maintain and review!
https://www.wikidata.org/wiki/Wikidata:Main_Page
are you saying wikidata now is full of LLM generated content?..
Define reviewable and maintainable
I've spent a career working with data and ontologies and I couldn't understand this readme, and the diagram only makes it more confusing.
I always giggle when software engineers use the term "ontology."
Was interested... but then realized it needs an NVIDIA api key?
Shameless plug: we built our own open source equivalent of auto-ontology a few months ago, that doesn't require any API key but your LLM one (or none if you use your coding agent subscription)
https://github.com/GetCassis/ontology-skills
will check it out! also thanks to the others for the clarifications.
I've not tried running it but while it defaults to NVIDIA hosted endpoints but the readme suggests you can host those yourself if you want.
You can self-host the LLMs involved, API key is only if you're using nvidia-hosted
We spend a small fortune trying to get ontology right as it's so critical to real world success of enterprise AI. Curious if this could actually help us.
I tapped out at the AI-isms in the readme.
Nvidia, I get you're all-in on AI, but please at least find a human to write some copy. It's both overly verbose and entirely lacking of important info at the same time.
I don't understand this ontology thing. What's the difference between this and a memory system plus skills to collect your enterprise's APIs and CLIs etc?
This is more about streamlining the Date Engineer / Data Scientist role. Business Intelligence, in other words.
Before: domain expert wants to ask a question about the data, but which of these hundred tables actually has the right ārevenue per Userā column? Ask a DE to help slice the data and bless the queries.
After: ask your agent. The agent looks at the ontology and can usually find the best blessed data. To enable this you need to write down all the tribal knowledge that lived in the DE/DS head; agents can take the first pass at proposing and updating metadata.
Under the covers the DE and DS roles switch to reviewing the underlying ontology, instead of working directly with business stakeholders on one-off queries.
well im just not gonna write down the tribal knowledge lmao
Ontology is a so-called white-box approach for AI: all dependencies are clearly stated and formalised. If something does not work as you wish, unlike LLMs, in ontology world you can clearly diagnose where the system fails. I understood the NVIDIA's idea this way: we take unreliable LLM and make a "reliable" ontology using it. If you look back into history, Prolog , was exactly a sort of "memory system" or, better put, a human-friendly database. Ontology is a further development of this concept.
What can an agent infer from this system that it can't infer from the codebase?
1. What is not in the codebase. Obvious of course, but building an ontology and maintaining it over time forces a company to document what was only living in the head of some experts and never written down. In the case of analytics use cases, think of all the quirks a data analyst might have in mind when querying the data. Very basic e.g. "exclude test accounts whenever you count customers". Every analyst on the team does it without thinking, it's rarely written anywhere, and an agent that doesn't know will happily count them.
2. Related to 1., your codebase or source material may hold different interpretations of the same definition (what "revenue" means can differ between Sales and Finance). The ontology forces you to choose one, or to state when to use each. Without that, the LLM may pick one a bit randomly.
3. It's a governance tool for humans. It gives the company a human-readable, "deterministic" source of truth to agree on what means what, then version it and evolve it.
One issue in a business is what does all the data actually mean? What are the various definitions and how do they relate to each other. What's a customer? Should that include people that have bought things, or anyone who is potentially buying? Do different systems use the same definitions, what are the columns, etc.
An ontology in this kind of context means a more precise machine readable spec of this and how it all relates to each other. You can totally build one by hand. The idea here is I think that it tries to figure it out itself and can take corrections over time, so you can ask natural language questions and get more reliable results rather than hoping an llm works out what "customers this month" means the same way each time over a wide span of data rather than having a much clearer API to work against.
This is what the Matt Pocock /grill-with-docs solves. It helps to build a ubiquitous language that helps with domain modelling. This along with the codebase seems to help agents a lot. I'm not clear on how the Nvidia tool here improves on this, or how it is more reliable?
A memory system is managed by the RL trained behavior, these systems use specific prompts and ordinary software to hardcode a policy. The RL version is usually more efficient, but it can't be improved or made more repeatable.
Open Source?
How about we start by open-sourcing the NVIDIA user-space libraries: libcuda, libcudart, libOpenCL, libnvtx, as well as nvidia-smi and the CUDA and OpenCL profilers.
Didn't expect Standard ML O_o
Nice is this Jensen Huang response's to Alex Karp recent statement on AI companies stealing people data with a subvention (token price) ? Because building ontologies is Palantir's core business.
The readme seems to be written for enterprise project managers. Does anyone know what philosophy or academic ontology work this ties to, if any? Iām having trouble understanding the specifics, Iāll be honestā¦
This isn't really anything to do with ontology in the philosophical or academic sense, outside of computer science.
In software, an ontology is just a machine-readable definition of the entities in a system - their names, properties, relationships etc. This tool analyzes a system and produces such a definition for agents to use.
how to use it
Game changer for entreprise (technical) debt
Enterprise like technical debt