Rendered at 16:23:55 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
sigpwned 10 hours ago [-]
The big questions I’m taking away are:
(1) they are claiming to produce apparently bijective closed-form symbolic representations/approximations of, among other things, LLMs. Is evaluating these closed-form representations more computationally efficient? The implications of that are potentially huge. It would be essentially analytic distillation. Fable on a chip and not a data center would be important — and disruptive - in many ways.
(2) Unsupervised, and even supervised, symbolic approaches to problem solving break down due to combinatorial explosion, among other things. This could potentially allow us to treat LLM training and inference as a search algorithm for novel symbolic approaches to solving new classes of complex problems hitherto unreachable through other approaches. If that works, I suspect it’s a feedback loop, too - the learnings from one representation push advances in the other. This would also increase the economic value of large training runs, since the model itself is now valuable, not just its inference.
(3) Per the above, can this push LLM design to greater capabilities?
The relationship between this and Anthropic’s J-space observation is also interesting. This is much, much deeper and more directly actionable, though.
EDIT: I ran my questions through Sonnet — yes, I appreciate the irony — and it was none too sanguine about questions (1) and (2), but thought (3) was reasonable. In any case, this is quite the paper. On reflection, I do think that the apparent reliance on very simple symbolic representations and tasks is underwhelming. But the approach is impressive. And obviously this is still early days, and the value of building a bridge between the very fuzzy LLM models and the rigorous, mechanically provable models would be enormous.
thesz 5 hours ago [-]
> "Second, there is no guarantee that a given neural network can be approximated by DISCOVER"
Page 7.
They train what appears as embeddings for outer product of roles and fillers. The role for language model can be a position in text, the filler can be an embedding of a word at that position. Then that matrix of a sum of these outer products is linearly mapped into NN encodings and then decoded by NN decoder.
The embeddings learned by this process are not necessarily smaller than original ones. Given that they participate in an outer product computation gives me impression that the resulting sum is much bigger than actual NN encoding, that is why it needs to be linearly mapped into NN encoding.
So, this paper will not necessarily lead to any computation savings.
But I am at page 6. ;)
sigpwned 3 hours ago [-]
Good find! But they stop short of saying it cannot be distilled to symbolic algebra. Regardless, your point stands. I scanned the paper in the middle of the night instead of sleeping. Clearly I didn’t do a close read! Thank you for pointing that out.
yorwba 6 hours ago [-]
Their representation is a key-value store that they embed via a tensor product to reconstruct the vectors the model produces. The catch is that it works best if the key-value mapping is task-specific. So if you already know how to solve the task the model is performing and can transform the input into a data structure that makes it easy to just read off the answer, you can probably also recover the model output.
That in itself is of course not super useful for tasks that we don't already know to do symbolically, but it's nice to see that they can manipulate the keys and values of their representation somewhat independently. So at least some of what LLMs do internally seems to involve a bit more structure than just linear combinations of some overcomplete dictionary.
The interesting question is whether this can be used to interpret the operation of a single layer by lifting it to operate on key-value stores instead, replacing matrix-vector products with reading from some keys and then writing to others. That could potentially move the balance of power more towards CPU inference instead of GPUs.
eli_gottlieb 2 hours ago [-]
> So at least some of what LLMs do internally seems to involve a bit more structure than just linear combinations of some overcomplete dictionary.
Well that's good to hear. Would have been rather exasperating to find out we could have somehow gotten Fable or Sol out of sufficiently advanced sparse-coding ages ago.
Oarch 6 hours ago [-]
It's a fascinating emergent behaviour but also one that could have been predicted?
I'd imagine that our brains have become similarly biased towards generating such a generalised symbolic structure over hundreds of millions of years of evolution?
I'm thinking how certain regions of the brain reliably represent things like motor skills, speech, abstract thought.
sigpwned 3 hours ago [-]
I agree. In retrospect, this seems almost inevitable. And our own minds at least to do some form of symbolic reasoning — literally language, which you are apparently capable of, dear reader, as a lower bound. There might be more symbolic reasoning in the conscious, unconscious, and subliminal parts of the mental workspace. I always hesitate to infer similarities between the brain/mind and LLMs, but I certainly track the comparison here.
riceflippa 5 hours ago [-]
> Is evaluating these closed-form representations more computationally efficient?
No it is way worse since computationally they are same things with symbolic having overhead of attending to the symbols.
sigpwned 3 hours ago [-]
My naive middle-of-the-night Claude question said the same thing. At least for this approach. I have not read the paper closely enough to refute you. But the concept of a lower-dimensional closed-form solution — which the paper seeks to discuss, please correct me if you read differently — is tantalizing, if only because it opens the path to different math, which can lead to optimization.
rwmj 5 hours ago [-]
Is it though? Brute force searching generally scales like 2^n. LLMs on the other hand can obviously be trained, and even though training is massively resource intensive, it's not 2^n.
virgil_disgr4ce 49 minutes ago [-]
Maybe someone can clear this up for me. I always thought it was already obvious that neural networks, artificial or otherwise, are symbol systems. Some input pattern reliably associated with some output pattern is, nearly by definition, a symbol. No? Am I just using the word symbol differently than researchers in these fields?
noduerme 7 hours ago [-]
Inference is just tokens transformed through a fixed crystalline structure, no? You already could put that on a chip. There's no particular reason it couldn't be represented as some set of symbolic equations instead of a layered process... it's just another kind of quantization.
When symbolic algorithms are that large, they're equally hard to reason with most of the time. The upshot would be a lot more storage required in exchange for more generalized computing, lessening the need for so much GPU in a lot of cases.
I don't see why a model couldn't be represented that way. After all, if you just polled the output of a model, you could evolve genetic algorithms to predict it with fairly high accuracy in a limited domain. Take that out to the Nth degree and you're basically just unspooling the model into a giant set of equations.
sigpwned 3 hours ago [-]
Yes, you certainly could put that on a chip. And people are doing it, for smaller models. The question is how big that chip would be for something like Fable, which is generally estimated in the trillions+ of parameters, and if, given the amount of memory, computation, and bandwidth required - at least with current methods, i.e., very high dimensional matrix math — a chip is the right way to go. Can you put a LM (just a smaller language model) on a chip? Empirically, yes, I believe it is done. Small enough ones probably even fit on a FPGA. Can you put a LLM on a chip? Depends on how L it is! My intuition says that some open smaller models might fit, like Haiku, but not Fable. But that’s just intuition talking.
The hope would be that this unlocks some substantially more efficient or parsimonious math that would fit better on a chip. And that’s clearly my words, not the authors’, per the comment above.
jsrozner 8 hours ago [-]
A big problem with some of these supervised* interpretability approaches is that they can find spurious structure. (There are lots of ways to make the model do what you want; which is roughly what Hewitt and Liang 2019 showed). This paper draws a contrast to a previous method, DAS (distributed alignment search) on page 20. These and related methods rest on theories of causal abstraction, which are great in theory, but harder in practice. DAS, for example, has faced numerous recent criticisms (Makelov 2024, Meloux 2025, Sutter 2025, Grant 2026, Kumon 2026). My favorite is the quite approachable Meloux et al.; Sutter 2025 is also really good, but relies on a sort of real number argument that allows a lossless encoding of every input.
My forthcoming paper at EMNLP offers an alternative that instead grounds the notion of representation in a very simple notion of the effect it has on model learning/behavior when you adversarially perturb it. For example, if I tell a model that in the context "I saw a duck quacking" it should replace 'duck' with 'glam', how much does it desire to replace 'duck' with 'glam' in "I need to duck out of the meeting" vs. "At the park a duck protected her ducklings." This method turns out to work quite well, and as we use only a single example, avoids the need for supervision.
The linked paper argues that their method, DISCOVER, is not supervised in the same way as DAS, since it does not directly optimize for causal effect. I have only skimmed this, but I am not so sure it might not suffer from a similar issue. They're still supervising to align representations with their underlying hypothesis, even if they don't directly supervise for causal outcomes.
Refs
- Hewitt and Liang 2019. Designing and interpreting probes with control tasks
- Kumon and Yanaka, 2026. Fine-grained analysis of shared syntactic mechanisms
- Meloux et al., 2025. Everything everywhere all at once
- Rozner and Shain 2026. Perturbation: A simple and efficient adversarial tracer for representation learning in LMs. https://arxiv.org/abs/2603.23821
- Sutter et al. 2025. The nonlinear representation dilemma
akie 6 hours ago [-]
So, basically you're saying: I have only skimmed the article, similar approaches had issues in the past, and my own method from my forthcoming paper is better.
I mean, no disrespect, but that's the core of your argument, yes?
Lerc 6 hours ago [-]
It certainly comes across as disrespectful.
To my reading the nature of supervision creating the structure seems to be the core of the argument.
akie 6 hours ago [-]
I have issue with the comment because he says he didn't read it, then unfavorably compares it to a previous method, and finally uses that negative review to plug his own article instead. His criticism might be valid, I'm not in a position to judge, but the self-promotion leaves a sour tastes in my mouth and makes me question how much of the criticism is just drummed up to make his own contribution appear more relevant.
cyberclimb 5 hours ago [-]
to his credit he did say he skimmed the paper, and it's honestly standard practice to do a first pass of skimming a paper before you'd go deeper into reading it anyways
smukherjee19 25 minutes ago [-]
I wonder if this paper has been peer-reviewed at a decent conference/journal.
jkingsman 11 hours ago [-]
The math and core experimentation here is beyond my abilities, but what I think I understand is that there are possible deeper patterns of representation that exist in LLMs that are distillations of core conceptual relations in grammar that we can get our heads around in a mathematical sense rather than apparent layer-smeared noise that somehow, un-interpretably (in a meaningful sense), resolve to correct grammar/inferences.
That's pretty cool. I hope I've got that kinda-right.
calebkaiser 11 hours ago [-]
I haven't read this in depth yet, though I plan to. If this general line of research is interesting to you, I'd recommend checking out some of the lines of research it touches upon--they're really rich and fascinating, and some are pretty approachable mathematically even if ML research papers aren't usually your thing. The related works section here seems pretty well stocked, but mechanistic interpretability is a pretty interesting peephole into this general vein: https://transformer-circuits.pub/
andytratt 9 hours ago [-]
distillation is now illegal tho
conmod278 10 hours ago [-]
[dead]
gps372 8 hours ago [-]
As I am going through the article, I was wondering why is this more interesting than having the ability to recover java programs from byte code. So I asked copilot the same question. It told me that - "Honestly this is where the difference between an engineer and researcher shows up!" .
riceflippa 5 hours ago [-]
it is not. in this case these researchers have lost their way, with how symbolics entered the conversation. to remind, it is via wanting to prove that ai vs traditional program is understanding deeper. well guess what, ai is not understanding your input better, its your mind playing tricks with language. For analogy, digital world is not real in a physical sense.
if output is food, and input is ingredients, then symbolic programs care about macro slicing dicing stacking them, while ai is micro level spice & heat that doesn't especially conscious to the ingredients, just that chemistry appears magical. hey! we eat our information food tho.
jackdoe 6 hours ago [-]
as long as it is honest, everything is ok.
gps372 5 hours ago [-]
It's very caring and reassuring also. It gave me an elaborated response on how researchers may discuss ridiculously fun theories. And what should be my takeaways as engineer. I guess I should turn off the Work IQ.
squidbeak 5 hours ago [-]
Reading stuff like this (as a layman), diminishing these things as 'Next token predictors' seems absurdly reductive. At some point we'll need to concede that 'selection' is a better term for this than prediction.
chrisjj 4 hours ago [-]
> diminishing these things as 'Next token predictors' seems absurdly reductive.
This shows a deep misunderstanding of the paper's claims, which in no way challenge the established view that these bots are next-token predictors.
Regardless, if all you want is a next-token selector, save your money and roll a die.
squidbeak 2 hours ago [-]
> This shows a deep misunderstanding of the paper's claims, which in no way challenge the established view that these bots are next-token predictors.
No, this shows an appreciation of the symbolic richness behind that token 'prediction' which the paper leads on.
> Regardless, if all you want is a next-token selector, save your money and roll a die.
Tell me, where is the emergent symbology guiding that dice?
lachlan_gray 2 hours ago [-]
For anyone like me who finds theory a little dense, you can prepend "quick" to an arxiv link to get a blogpost-style summary
e.g.
quickarxiv.org/abs/2608.29530
urbnspacecowboy 10 minutes ago [-]
N.B. quickarxiv.org just redirects to alphaxiv.org, so the following (i.e. change "ar" to "alpha") works just as well:
The human mind cannot comprehend the capacity of massively multidimensional space.
Just going from 2D to 3D creates massive new positional potential (e.g. surface of the earth, vs. the atmosphere above earth...).
Now imagine 1,000 dimensions.
larodi 9 hours ago [-]
Imagine a box of balls. They have size, weight, colour, density… etc. These properties, each a measure, are dimensions and they are orthogonal to each other. Taken together are multi-dimensional.
qsera 9 hours ago [-]
Now take a set of words. They have "sizeness", "weightness", "colorness" and "densityness"...and "pythonness" and "haskellness" and even "adjectiveness" and "verbness" and so on and so forth...You can readily see that this can encode arbitrary patters (like language grammer or program syntax)
Training identifies these dimensions in the training data and links it with each word/token. Then given a stream of such tokens, each with its own set of dimensions (which can be huge), and LLM predicts the dimensions that the next token is most likely to have...
larodi 3 hours ago [-]
my take from all these is that is perhaps possible to teach similarity search to 5th graders. not an elementary teacher myself, but many math concepts are much easier to explain with allegories (visual and other) than with formalism. a good example perhaps is what Mamikon does in his visual explanations of calculus (https://calteches.library.caltech.edu/4007/1/Calculus.pdf)
formalisms come only after there is understanding of the phenomena and a specific need to communicate concepts across members of community - in a concise and commonly comprehensive way. my understanding is that education often fails to train understanding and jumps directly to formalisms. and this is right also for STEM schools, where it is presumed everyone has some innate inclination to the matter. but this presumption, in my experience as university teacher, is not entirely correct, and students are not universally prepared to express lack of understanding, and particularly when it is expected from them to rather indicate understanding.
it seems very weird, the concept, to reward the correct answer, while suppressing and discouraging the expression of misunderstanding or confusion. why is weird? because learning comes trough trial and error, and the error part is half of it. so we need to encourage people to express their errors or lack of understanding, so that we can then fill in the gaps and eventually stand on a common ground. sadly, many forums, HN included, does not cherish misunderstanding. our posts here mostly get upvoted whenever they cross a sort of threshold about the amount of truth in our writing, and not because of daring expressions of confusion.
hliyan 9 hours ago [-]
This is nonsense. The human mind cannot visualize more than 3 dimensions. It can perfectly comprehend any number of dimensions as long as they are represented in a vector space. In fact, that's what linear algebra does.
NitpickLawyer 9 hours ago [-]
I think you accidentally a word, there. GP is talking about comprehending the capacity of massively multi-dimensional space.
On the other hand I deal with higher dimensions better than most, possibly because of my aphantasia.
philipswood 7 hours ago [-]
I think it would be fair to say that at least a few people can do a small amount of 4d visualisation.
Agreed, I don't think anyone has a good grasp on, say, 64K D, but we can do a bit better than just 3.
taneq 6 hours ago [-]
I’ve heard people claiming that they can but I’ve never heard compelling evidence that they’re directly visualising 4D objects rather than 3D projections of them, or some symbolic representation (arrays of numbers etc.)
Paradigma11 5 hours ago [-]
So it is like 3D because you do not visualize the complete 3D models but only 2D projections of them. Or do you really visualize the back and inside of a vase at the same time?
taneq 3 hours ago [-]
If I visualise a 3D object then it’s in at least pseudo-3D (as in front surface with depth) but I can switch it to a kinda volumetric view. This might be thanks to years worth of 3D modeling and CAD which helped me map between first person and third person cameras and between solid and transparent, though. 4D doesn’t even make sense though, like I can do an animated 3D volume?
sublinear 9 hours ago [-]
Yes. It's worth pointing out that anything with n distinct parameters is just a point in n-dimensional space. We're so used to handling so many dimensions that nobody ever bats an eye until someone brings up the magic word "dimensions". It's quite intuitive actually.
The trivial example that comes to mind is the character customization sliders in many video games.
qsera 10 hours ago [-]
>The human mind cannot comprehend the capacity of massively multidimensional space.
That is why the scam works, because investors are humans...
left-struck 10 hours ago [-]
Which scam sorry?
qsera 10 hours ago [-]
The scam that is based on the implicit claim that LLM is a path to AGI.
Seeing LLMs for what they really are will also make it clear they are fundamentally unfit for a lot of tasks they are currently marketed for...
kleiba2 9 hours ago [-]
Too broad a statement, and without substantiation, to be taken serious, sorry.
Madmallard 8 hours ago [-]
Is it?
They're borderline useless and certainly potentially inadvertently malicious for writing, customer service, speech to text, writing large amounts of complex code, therapy, medical diagnostics... the list goes on
Sounds like you're part of the problem?
It's really a serious problem just eroding the fabric of society in real-time. Being complacent in it or believing in the promise is just wholly foolish and bad for everyone.
sublinear 9 hours ago [-]
Too shallow of a dismissal, and you don't determine what everyone else takes seriously.
It's been several years now of LLMs only appeasing those with low expectations and inexperience. Unless the only goal was generating boilerplate or really sloppy proofs of concept, LLMs are a waste time for everyone else. This argument is so over already. We're all just hoping for a soft landing when the hangover really kicks in.
dboreham 7 hours ago [-]
I have 40+ years experience and extremely high standards. What you say is entirely wrong.
qsera 6 hours ago [-]
>I have 40+ years experience
What domain?
brabel 9 hours ago [-]
You are disconnected from reality. The whole industry is already completely dominated by LLMs generating code. Bury your head in the sand all you want. This is not about low expectations or inexperience at all. Your condescending tone doesn't make you look smarter, it makes you look like an Amish who expects the industrial revolution is temporary and soon people will come to their senses and stop using all this nonsense industrial technology.
Ariarule 1 hours ago [-]
A sidenote, but that is not a typical Amish opinion of technology:
People are using them because they're being shoved down their throats and they're complacent.
Software quality, maintainability, exploitability, morale, competency are all at all-time lows and just worsening.
It's really bad to defend this.
imtringued 7 hours ago [-]
I spent the last 6 months reviewing all of the AI generated code primarily with gpt 5.3 codex but after I bought the subscription I also used gpt 5.5 and although I was constantly rejecting the bad code and I was mostly happy with the end result.
Then they removed the approve everything manually mode and I thought, maybe I'm the one out of touch, I should try vibe coding. I gave it a big spec of things to implement and I was pretty vague on the details, but I was kind of hoping that the AI would get it right anyway.
I noticed that when you let the AI just do its thing, you spend a lot of time staring at the screen waiting for it to finish. No wonder all these people are obsessed with parallel agents.
Anyway, after a while it produced the end result and it was pretty bad and I don't even mean the code. This was with gpt 5.6 Terra. Definitively not the type of "did a week's worth of work in one hour" type of deal.
Meanwhile if you guide it with your own technical skills the end result is a tireless code generator that operates at 80% of the quality of your own code but is only roughly 1.5x to 2x faster than doing it yourself. The biggest multipliers come from making the AI do things humans don't want to do, like writing a bunch of thorough tests but those don't translate into increased productivity, the opposite in fact. They translate into higher software quality but if you vibe the tests and don't review them, even that disappears.
vandopereira 50 minutes ago [-]
to avoid lot os problems i use kaplira to help maintain de code in boundaries and not let the agent do everything they want.
Madmallard 6 hours ago [-]
yes i have written a jillion lines with AI as well
i'm a worse programmer now because of it
pineaux 9 hours ago [-]
What about the mathematical advancements?
sublinear 9 hours ago [-]
Doesn't that say more about the massive crumb tray nobody ever bothered to empty at the bottom of mathematics?
I'm sure someone will point out something like the 4-color theorem as a counterargument. Where is that kind of theorem proving in this generation of AI? We seem to have hit a dead end rather quickly.
khalic 7 hours ago [-]
Lol, are you saying the Erdos problems are a "crumb" that nobody bothered to empty? Are you doing a comedy routine?
4b11b4 11 hours ago [-]
Sounds reasonable... That the model is sometimes learning a lossy vector representation of something symbolic in nature... Sure, a NN can approximate a function?
They say this holds in... Some examples they found?
I don't enough about this area
andytratt 9 hours ago [-]
yep
trnkinju 8 hours ago [-]
Symbolism has tried to strike back repeatedly ever since statistical learning revived with AlexNet. With all the due respect one can have for the names Smolensky and Linzen from the perspective of linguistics, the question about the applicability, generalizability and robustness of the method proposed here should be raised. It seems from section 3.5 of the paper that one cannot be so optimistic about it at least as yet. I get it that the method is still in its infancy, but we've already got the kind of Mech Interp as pushed forward by Neel Nanda and co, among other lines of research. Not that we are forced to make a choice between all interpretability works, or this TPR method is inherently inferior to the other ones, but we can be moderately cautious when looking at such progress.
9 hours ago [-]
0xdeadbeefbabe 11 hours ago [-]
It's like Neo says "You get used to it, though. Your brain does the translating. I don't even see the code." He was referring to something like a K, Q, V vector at the time I believe.
monster_truck 11 hours ago [-]
Cypher says that, and he's clearly referring to a blonde, a brunette, and a redhead.
Bluestein 8 hours ago [-]
AND then they walk by the frame, next shot.-
andytratt 9 hours ago [-]
lol
andytratt 9 hours ago [-]
this is an obvious result. for example, this guy has been writing on substack about this for at least a year or two (with code snippets) explaining the phenomenon of grokking and the ghostbasin.com concept - https://richardaragon.substack.com/
their algorithm is even named "DISCOVER" so they set out to discover the connective tissue of why the universe has invariants like math, and lo it was discovered.
i guess good job for having credentials & publishing the math so people 2years behind the curve can learn from your tenure?
yes. large matrices can gradient descend to understand arbitrary symbolic logic.
ENGLISH IS INSUFFICIENT but it is at least a few decades of math proofs & progress :) welcome to the future Slackernews
subsistence234 8 hours ago [-]
Post the actual articles that you have in mind. What I've seen is vague slop.
A good example is https://richardaragon.substack.com/p/a-universal-prime-funct... describing a supposed "universal prime function" which is simply a finite approximation using a sum of 50 sines (each applied to a linear term plus a sine-log offset). The 53 parameters are fitted to the first 10^3 or so prime numbers. This is followed by the *absolutely ridiculous* claim that if the function approximates the first 10^3 primes well, it must also fit the remaining prime numbers (of which there are infinitely more than 10^3000000000) equally well.
Then they suggest "A formal proof connecting this function to the RH would involve the following steps" using this great discovery: "1. Correspondence with the Explicit Formula: Demonstrate that the oscillatory correction term in our function corresponds to the sum over zeta zeros in the explicit formula for ψ(x) or π(x). 2. Error Bound: Prove that the error in the prime counting function derived from our function is bounded by O(√x log x). 3. Contradiction: Show that if any non-trivial zero were to lie off the critical line ℜ(s) = 1/2, the error would exceed the bound, leading to a contradiction."
This isn't even midwit math.
It's the kind of naive ideas I had as a high schooler, who was good at high school math and who knew how to code functions and plots in Mathematica, but who had no understanding of higher math. This kind of naive approach to RH signals that one doesn't even understand the problem.
colordrops 9 hours ago [-]
"Vectors seem inadequate for capturing the structure of language, logic, and other cognitive domains, yet neural networks achieve impressive performance in these areas". Missing the forest for the trees? Aren't neural networks modeled after biological systems? Our brains are obviously able to contain symbolic structure despite not having a "symbol processing unit".
suddenlybananas 8 hours ago [-]
People really overstate the relationship between ANNs and the brain, they have very different mechanisms and only have a similarity if you squint at 100000 feet. ANNs don't have neurotransmitters or even action potentials.
azakai 2 hours ago [-]
But neurotransmitters and action potentials don't help in modeling symbolic structure.
That is, yes, ANNs are not brains. There are countless differences. But are there differences at the computational level? ANNs are meant to model brain computation, not brain biology.
(There is still a lot to debate there, I'm not saying "ANNs are perfect computational models for the brain")
suddenlybananas 30 minutes ago [-]
>But neurotransmitters and action potentials don't help in modeling symbolic structure.
This is exactly why Fodor argues that psychology should be explained on its on level with symbols rather than appealing to neurology. But if you're interested in modeling symbols, there's much better options than ANNs (see nearly any programming language ever).
>ANNs are meant to model brain computation
But we don't really know how that works! So if you know if you're not modelling the low-level behaviour right, you can't assume that there's a correspondence of the higher level computation when you don't really know what that higher level computations are.
azakai 1 minutes ago [-]
We don't know all the details about how the brain computes, you are right.
But we do have a hypothesis: that it is done by a large number of simple units with very high connectivity and in deep layers. This is what neural networks model.
Personally I was skeptical of this model of the brain, but they have achieved remarkable success in practice, as well as Nobel prizes. The neural networks people may have been onto something all along (I say that grudgingly).
addag 3 hours ago [-]
While I agree with this statement, one could argue that what's important is not much the support but the emergent properties - in the same way that a wave is still a wave whether it is in the water or in the air.
Thus putting things similar to neurons in a network and making them able to learn could create behaviors similar to the brain. The fact that that biology used chemistry + electrical signal and computers use ReLU-like activation could be merely choosing the most efficient way to enable training.
imtringued 7 hours ago [-]
Correct. Worst of all, even if you do build a spiking neural network, the update rule is kind of a mystery. To have a good update rule, a biological neuron needs to be kind of like a tiny computer in its own right. You might be able to model synapses as weights between neurons, but the neuron carries further internal states within in itself and how the "update rule" uses those internal states is not known at all.
wetpaws 8 hours ago [-]
[dead]
andytratt 9 hours ago [-]
yep
antonvs 9 hours ago [-]
I hate that whole intro - the first four sentences - so much. It’s nothing but unsupported assumptions. Basically, a strawman that they can do battle with in the paper. Not an auspicious start.
suddenlybananas 8 hours ago [-]
These aren't really strawmen, they're more or less than mainstream opinion in the cognitive sciences from the 80s to maybe 2015-2020 or so.
jephs 4 hours ago [-]
Paul Smolensky is a cognitive science titan from that era. He worked with Hinton, Rumelhart, and McClelland on parallel distributed processing, and literally wrote the book on tensor product representations in cognition, with Geraldine Legendre: https://mitpress.mit.edu/9780262516198/the-harmonic-mind-vol...
He's the axis of this particular group of researchers, being the most senior at the place where they all met, Johns Hopkins.
So this is less a straw man and more a quick reminder to his peers: "Right, so, remember this particular thread we've spent the last 40 years hashing out, here we've got another contribution to that particular conversation."
andytratt 9 hours ago [-]
got to top 2 HN tho lol
unjuno 3 hours ago [-]
[dead]
hn1rig3rak 3 hours ago [-]
[dead]
ozereray1 7 hours ago [-]
[dead]
KashifBuilds 7 hours ago [-]
[flagged]
addag 3 hours ago [-]
From the abstract "Further, our symbolic approximation allows us to modify an LLM's behavior in targeted ways via precise interventions on its internal representations [...]".
If this is true and easily computable, this might have big impact in AI safety, as it seems to be really lacking today.
(1) they are claiming to produce apparently bijective closed-form symbolic representations/approximations of, among other things, LLMs. Is evaluating these closed-form representations more computationally efficient? The implications of that are potentially huge. It would be essentially analytic distillation. Fable on a chip and not a data center would be important — and disruptive - in many ways.
(2) Unsupervised, and even supervised, symbolic approaches to problem solving break down due to combinatorial explosion, among other things. This could potentially allow us to treat LLM training and inference as a search algorithm for novel symbolic approaches to solving new classes of complex problems hitherto unreachable through other approaches. If that works, I suspect it’s a feedback loop, too - the learnings from one representation push advances in the other. This would also increase the economic value of large training runs, since the model itself is now valuable, not just its inference.
(3) Per the above, can this push LLM design to greater capabilities?
The relationship between this and Anthropic’s J-space observation is also interesting. This is much, much deeper and more directly actionable, though.
EDIT: I ran my questions through Sonnet — yes, I appreciate the irony — and it was none too sanguine about questions (1) and (2), but thought (3) was reasonable. In any case, this is quite the paper. On reflection, I do think that the apparent reliance on very simple symbolic representations and tasks is underwhelming. But the approach is impressive. And obviously this is still early days, and the value of building a bridge between the very fuzzy LLM models and the rigorous, mechanically provable models would be enormous.
They train what appears as embeddings for outer product of roles and fillers. The role for language model can be a position in text, the filler can be an embedding of a word at that position. Then that matrix of a sum of these outer products is linearly mapped into NN encodings and then decoded by NN decoder.
The embeddings learned by this process are not necessarily smaller than original ones. Given that they participate in an outer product computation gives me impression that the resulting sum is much bigger than actual NN encoding, that is why it needs to be linearly mapped into NN encoding.
So, this paper will not necessarily lead to any computation savings.
But I am at page 6. ;)
That in itself is of course not super useful for tasks that we don't already know to do symbolically, but it's nice to see that they can manipulate the keys and values of their representation somewhat independently. So at least some of what LLMs do internally seems to involve a bit more structure than just linear combinations of some overcomplete dictionary.
The interesting question is whether this can be used to interpret the operation of a single layer by lifting it to operate on key-value stores instead, replacing matrix-vector products with reading from some keys and then writing to others. That could potentially move the balance of power more towards CPU inference instead of GPUs.
Well that's good to hear. Would have been rather exasperating to find out we could have somehow gotten Fable or Sol out of sufficiently advanced sparse-coding ages ago.
I'd imagine that our brains have become similarly biased towards generating such a generalised symbolic structure over hundreds of millions of years of evolution?
I'm thinking how certain regions of the brain reliably represent things like motor skills, speech, abstract thought.
No it is way worse since computationally they are same things with symbolic having overhead of attending to the symbols.
When symbolic algorithms are that large, they're equally hard to reason with most of the time. The upshot would be a lot more storage required in exchange for more generalized computing, lessening the need for so much GPU in a lot of cases.
I don't see why a model couldn't be represented that way. After all, if you just polled the output of a model, you could evolve genetic algorithms to predict it with fairly high accuracy in a limited domain. Take that out to the Nth degree and you're basically just unspooling the model into a giant set of equations.
The hope would be that this unlocks some substantially more efficient or parsimonious math that would fit better on a chip. And that’s clearly my words, not the authors’, per the comment above.
My forthcoming paper at EMNLP offers an alternative that instead grounds the notion of representation in a very simple notion of the effect it has on model learning/behavior when you adversarially perturb it. For example, if I tell a model that in the context "I saw a duck quacking" it should replace 'duck' with 'glam', how much does it desire to replace 'duck' with 'glam' in "I need to duck out of the meeting" vs. "At the park a duck protected her ducklings." This method turns out to work quite well, and as we use only a single example, avoids the need for supervision.
The linked paper argues that their method, DISCOVER, is not supervised in the same way as DAS, since it does not directly optimize for causal effect. I have only skimmed this, but I am not so sure it might not suffer from a similar issue. They're still supervising to align representations with their underlying hypothesis, even if they don't directly supervise for causal outcomes.
Refs
- Hewitt and Liang 2019. Designing and interpreting probes with control tasks
- Kumon and Yanaka, 2026. Fine-grained analysis of shared syntactic mechanisms
- Meloux et al., 2025. Everything everywhere all at once
- Rozner and Shain 2026. Perturbation: A simple and efficient adversarial tracer for representation learning in LMs. https://arxiv.org/abs/2603.23821
- Sutter et al. 2025. The nonlinear representation dilemma
I mean, no disrespect, but that's the core of your argument, yes?
To my reading the nature of supervision creating the structure seems to be the core of the argument.
That's pretty cool. I hope I've got that kinda-right.
This shows a deep misunderstanding of the paper's claims, which in no way challenge the established view that these bots are next-token predictors.
Regardless, if all you want is a next-token selector, save your money and roll a die.
No, this shows an appreciation of the symbolic richness behind that token 'prediction' which the paper leads on.
> Regardless, if all you want is a next-token selector, save your money and roll a die.
Tell me, where is the emergent symbology guiding that dice?
e.g. quickarxiv.org/abs/2608.29530
https://www.alphaxiv.org/abs/2608.29530
Just going from 2D to 3D creates massive new positional potential (e.g. surface of the earth, vs. the atmosphere above earth...).
Now imagine 1,000 dimensions.
Training identifies these dimensions in the training data and links it with each word/token. Then given a stream of such tokens, each with its own set of dimensions (which can be huge), and LLM predicts the dimensions that the next token is most likely to have...
formalisms come only after there is understanding of the phenomena and a specific need to communicate concepts across members of community - in a concise and commonly comprehensive way. my understanding is that education often fails to train understanding and jumps directly to formalisms. and this is right also for STEM schools, where it is presumed everyone has some innate inclination to the matter. but this presumption, in my experience as university teacher, is not entirely correct, and students are not universally prepared to express lack of understanding, and particularly when it is expected from them to rather indicate understanding.
it seems very weird, the concept, to reward the correct answer, while suppressing and discouraging the expression of misunderstanding or confusion. why is weird? because learning comes trough trial and error, and the error part is half of it. so we need to encourage people to express their errors or lack of understanding, so that we can then fill in the gaps and eventually stand on a common ground. sadly, many forums, HN included, does not cherish misunderstanding. our posts here mostly get upvoted whenever they cross a sort of threshold about the amount of truth in our writing, and not because of daring expressions of confusion.
On the other hand I deal with higher dimensions better than most, possibly because of my aphantasia.
Agreed, I don't think anyone has a good grasp on, say, 64K D, but we can do a bit better than just 3.
The trivial example that comes to mind is the character customization sliders in many video games.
That is why the scam works, because investors are humans...
Seeing LLMs for what they really are will also make it clear they are fundamentally unfit for a lot of tasks they are currently marketed for...
They're borderline useless and certainly potentially inadvertently malicious for writing, customer service, speech to text, writing large amounts of complex code, therapy, medical diagnostics... the list goes on
Sounds like you're part of the problem?
It's really a serious problem just eroding the fabric of society in real-time. Being complacent in it or believing in the promise is just wholly foolish and bad for everyone.
It's been several years now of LLMs only appeasing those with low expectations and inexperience. Unless the only goal was generating boilerplate or really sloppy proofs of concept, LLMs are a waste time for everyone else. This argument is so over already. We're all just hoping for a soft landing when the hangover really kicks in.
What domain?
"Amish do not view technology as evil, and individuals may petition for acceptance of a particular technology in the local community" - https://en.wikipedia.org/wiki/Amish_way_of_life#Use_of_moder...
See also https://thereader.mitpress.mit.edu/how-the-amish-let-technol... about some Amish use of cell phones with adapters to be more payphone/landline-like and https://news.ycombinator.com/item?id=21349989 was a discussion about the use of pneumatic machinery by the Amish.
Maybe you forgot that important tidbit?
People are using them because they're being shoved down their throats and they're complacent.
Software quality, maintainability, exploitability, morale, competency are all at all-time lows and just worsening.
It's really bad to defend this.
Then they removed the approve everything manually mode and I thought, maybe I'm the one out of touch, I should try vibe coding. I gave it a big spec of things to implement and I was pretty vague on the details, but I was kind of hoping that the AI would get it right anyway.
I noticed that when you let the AI just do its thing, you spend a lot of time staring at the screen waiting for it to finish. No wonder all these people are obsessed with parallel agents.
Anyway, after a while it produced the end result and it was pretty bad and I don't even mean the code. This was with gpt 5.6 Terra. Definitively not the type of "did a week's worth of work in one hour" type of deal.
Meanwhile if you guide it with your own technical skills the end result is a tireless code generator that operates at 80% of the quality of your own code but is only roughly 1.5x to 2x faster than doing it yourself. The biggest multipliers come from making the AI do things humans don't want to do, like writing a bunch of thorough tests but those don't translate into increased productivity, the opposite in fact. They translate into higher software quality but if you vibe the tests and don't review them, even that disappears.
i'm a worse programmer now because of it
I'm sure someone will point out something like the 4-color theorem as a counterargument. Where is that kind of theorem proving in this generation of AI? We seem to have hit a dead end rather quickly.
They say this holds in... Some examples they found?
I don't enough about this area
their algorithm is even named "DISCOVER" so they set out to discover the connective tissue of why the universe has invariants like math, and lo it was discovered.
i guess good job for having credentials & publishing the math so people 2years behind the curve can learn from your tenure?
yes. large matrices can gradient descend to understand arbitrary symbolic logic.
ENGLISH IS INSUFFICIENT but it is at least a few decades of math proofs & progress :) welcome to the future Slackernews
A good example is https://richardaragon.substack.com/p/a-universal-prime-funct... describing a supposed "universal prime function" which is simply a finite approximation using a sum of 50 sines (each applied to a linear term plus a sine-log offset). The 53 parameters are fitted to the first 10^3 or so prime numbers. This is followed by the *absolutely ridiculous* claim that if the function approximates the first 10^3 primes well, it must also fit the remaining prime numbers (of which there are infinitely more than 10^3000000000) equally well.
Then they suggest "A formal proof connecting this function to the RH would involve the following steps" using this great discovery: "1. Correspondence with the Explicit Formula: Demonstrate that the oscillatory correction term in our function corresponds to the sum over zeta zeros in the explicit formula for ψ(x) or π(x). 2. Error Bound: Prove that the error in the prime counting function derived from our function is bounded by O(√x log x). 3. Contradiction: Show that if any non-trivial zero were to lie off the critical line ℜ(s) = 1/2, the error would exceed the bound, leading to a contradiction."
This isn't even midwit math.
It's the kind of naive ideas I had as a high schooler, who was good at high school math and who knew how to code functions and plots in Mathematica, but who had no understanding of higher math. This kind of naive approach to RH signals that one doesn't even understand the problem.
That is, yes, ANNs are not brains. There are countless differences. But are there differences at the computational level? ANNs are meant to model brain computation, not brain biology.
(There is still a lot to debate there, I'm not saying "ANNs are perfect computational models for the brain")
This is exactly why Fodor argues that psychology should be explained on its on level with symbols rather than appealing to neurology. But if you're interested in modeling symbols, there's much better options than ANNs (see nearly any programming language ever).
>ANNs are meant to model brain computation
But we don't really know how that works! So if you know if you're not modelling the low-level behaviour right, you can't assume that there's a correspondence of the higher level computation when you don't really know what that higher level computations are.
But we do have a hypothesis: that it is done by a large number of simple units with very high connectivity and in deep layers. This is what neural networks model.
Personally I was skeptical of this model of the brain, but they have achieved remarkable success in practice, as well as Nobel prizes. The neural networks people may have been onto something all along (I say that grudgingly).
Thus putting things similar to neurons in a network and making them able to learn could create behaviors similar to the brain. The fact that that biology used chemistry + electrical signal and computers use ReLU-like activation could be merely choosing the most efficient way to enable training.
He's the axis of this particular group of researchers, being the most senior at the place where they all met, Johns Hopkins.
So this is less a straw man and more a quick reminder to his peers: "Right, so, remember this particular thread we've spent the last 40 years hashing out, here we've got another contribution to that particular conversation."
If this is true and easily computable, this might have big impact in AI safety, as it seems to be really lacking today.