This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
I made a comment recently speculating that Terence Tao may be a paid promoter of AI. After some off-site discussion elsewhere, our resident shaman on whom the mods cast a long-duration silence posted my comment as a circus exhibit on Twitter to demonstrate a dangerous form of psychosis. Fortunately, OpenAI deemed this a worthy time to intervene and publish an ad featuring Terence Tao, who has taken time out of his busy schedule to assist this struggling non-profit with their promotional.
I want to dissect what I think is really going on.
For starters, every time I sit down to watch WoW on YouTube, I'm greeted with my favourite streamer telling me about the super fun game Raid Shadow Legends, which I should definitely download and play because it's super fun. Is the OpenAI-Terence Tao relationship like this? Not really. Terence Tao does appear to actually spend some time playing around with LLMs. Further, he's not exactly saying a bunch of empty marketing blather, either. In fact, probably to the annoyance of some readers here, I want to take a couple paragraphs for a technical aside, because there is actually subtlety here. I'll bound this in horizontal bars so non-technical readers can skip it:
The main talking point is that automated theorem proving is a perfect fit for LLMs precisely because it's not vulnerable to their main catastrophic failure mode: hallucination. The model can hallucinate whatever it wants, but the text still goes into the theorem prover, and if it's bullshit, well, the prover just rejects it and you query the LLM again. Do this in a loop, burn whatever unholy amount of compute you want, and if the loop stops, you've got yourself a proof! (Well, or a bug in the theorem prover. Or a "You've run out of tokens on your budget" error message. But I digress). This story is largely true. There's a giant asterisk of "Uh, so how much compute we talkin' about?", and the answer is "As much as you need or can afford, whichever comes first!" Which is, of course, the business model.
I will point out one additional technical nit-pick that annoys me because Terence Tao is working in Lean 4, which is a dependently-typed theorem prover. In classical mathematics, one is concerned solely with whether a theorem is true or false, and the structure of the proof is basically irrelevant as long as it's valid. Lean 4 is not based on this model. Rather, it's based on a more computationally-motivated model of mathematics pioneered by Brouwer in the early 1900s called "constructivism." In this world, the question isn't the boolean notion of "Is this theorem true or false?" but rather the related but distinct notion of "Which proof do you have?" To ground this in practical terms, consider the following example: I can prove that True|False and Yes|No are isomorphic, but I can do so in multiple ways: I can map True to Yes and False to No, or I can map True to No and False to Yes (and then show that there are respective inverses which preserve identity, obviously). It is in this sense that one can meaningfully say "Which proof of isomorphism?" when I say I have a proof of isomorphism. Perhaps this all sounds like technobabble, but to connect it to the preceding paragraph: you can immediately see how this does reveal some cracks into the narrative being sold there. It does actually matter which proof is produced, not merely in a social sense of "can any human understand this wall of text the LLM spit out", but in a technical, computationally-relevant sense. For pure mathematics, this distinction is often not considered important -- in fact, many classical mathematicians aren't even aware of the difference, and will be confused if you try to explain it to them and think this is all a bit silly. However, it's not a silly or minor distinction for the following reason: one of the motivations of this computational model for "theorem provers" (it's really a programming language + compiler, rebranded for mathematicians) like Lean is so that formal methods can be applied not just to classical mathematics, but to software in general. And as soon as you enter software formalisation, this distinction is no mere intellectual curiosity, but of paramount relevance. The classical-style logic in the preceding paragraph does not apply to constructivist logic used for software formalisation! I'm sure this distinction is not lost on Terence Tao. But that doesn't concern OpenAI. OpenAI is more concerned with whether the distinction will be lost on the MBAs listening to Terence Tao, and the answer is "absolutely."
Ok, no more technical details like that, I promise. Back to the social level:
So, I mentioned Raid Shadow Legends is a poor metaphor for the OpenAI-Tao relationship. Let me propose some better ones: Michael Phelps and Wheaties (with the added benefit that Terence Tao never smokes weed. See, this is why mathematicians are better than athletes), or better yet, attending Harvard University. This may seem like a strange juxtaposition, but I've done so intentionally, because the marketing is obvious in one but subtle in the other, but it's actually the same trick: the goal is to misattribute performance. With Wheaties, the goal is to sell the notion that Michael Phelps is a great swimmer because he has a healthy diet of stuff like Wheaties, and if you eat Wheaties, maybe you'll perform well, too! Of course, in reality, he was eating sugar-coated french toast and chocolate chip pancakes because he needed 10k Calories/day just to break even on energy, and the reason he's such an awesome swimmer is in large part genetics. Wheaties, or anything similar to it, has virtually no relevance to Michael Phelps at all. But what about Harvard?
Well, Harvard sells the image so well that most people outside this forum outright believe the illusion. The illusion is, of course, that attending Harvard makes you smart and likely to succeed, rather than Harvard accepting only people who were smart and likely to succeed in the first place and thus redirecting credit for these future achievements to Harvard. Mark Zuckerberg may see Harvard as a pointless waste of time, but the world sees Harvard as "The university that made Mark Zuckerberg happen!"
I like the Harvard analogy because this is surely the intent with Terence Tao. There's a high chance sooner or later Terence Tao will prove something cool "using" ChatGPT, and if he does, it would be really awesome if we could make it sound like the secret ingredient in the ChatGPT-Terence Tao alliance was ChatGPT, when obviously the actual secret ingredient is Terence Tao. The analogy I always use for this is stone soup, a European folktale where starving travelers dupe gullible townsfolk into helping them make soup from stones by requesting "extra" ingredients bit by bit until they've just made actual soup, thus astonishing the gullible townsfolk.
There are a lot of other things I could say on this, especially on the technical side, as there are a lot of clever tricks you can pull to make it look like a model is doing more than it really is, but I'll stop for now and conclude with this:
Just be cognizant that OpenAI, and all the other LLM vendors, do marketing. They have an enormous budget that dwarfs anything you have ever seen before. Remember the reality distortion field of Black Lives Matter? Or trans people? Imagine that, but like... two orders of magnitude larger. That is the level of persuasive pressure we're dealing with here.
Just take it all with a grain of salt.
The main thing about constructivists is that they do not generally accept proofs where you you assume the negation of what you are want to prove and arrive at a contradiction (Reductio ad absurdum -- though it seems that some cases remain valid in intuitionist/constructivist logic).
As a layman, my gut feeling is that the constructivists would probably accept Euclid's proof of an infinite magnitude of primes, because it gives you a rather concrete algorithm for finding a prime which is not in your list of all primes, but not Hilbert's proof of his Nullstellensatz, because it does not give you a way to determine the algebraic relationship.
WP on constructivism
Moving on further
I am not sure I follow. Suppose you have a program and a claim about that program ("this Turing machine will halt for any input", "This C code will never invoke undefined behavior", etc). In my mind, a proof which is just half a gigabyte of gibberish without rhyme or reason would still be acceptable for such practical problems. If someone formally verified that a given version of the Linux kernel did not suffer from a given class of exploits, I would not complain about them using proof by negation.
By contrast, open problems in mathematics are not things where our main interest is in knowing the answer. Few people are interested in P==NP because they think there is a practical algorithm for solving SAT to be discovered. If an ASI told us what the answer is ("P is equal to NP, but the polynomial has coefficients and exponents so large that your mathematics can't even express them using all the protons in the universe"), that would be of little value. People are interested in these big open questions because their answer sometimes lead to the development of new and exciting branches of mathematics.
The textbook example of a problem which looked promising in that regard until it was proven would be the four color theorem. "So it can just be proven by brute-forcing 1834 configurations with a computer? Seems it was not a nice problem, after all."
With most of our existing software, the problem is even identifying what formal properties we would want, or constructing it so that has the desired properties (and ideally we can easily proof them).
The Hilbert thing is more a sign of the times. Back then, there was a lot of resistance and emotional slapfighting over this, because in ye olden days, people were very concerned about which postulates were True (TM), and viewed any attempt to discuss this as an attack on Truth.
Today, we worry less about this absolute notion of Truth and more about models: is Euclid's parallel postulate True or False? Well, there's actually nothing to fight over: you just get different geometries, but they're all meaningful and useful! So rather than saying "The parallel postulate is true, therefore XYZ", you can just say "When the parallel postulate is true, XYZ holds." Even questioning logical primitives that seem "very true" like "can you appeal to a theorem more than once in the same proof?" turns out to have surprisingly useful implications! For example, if you're modeling a cookie, it makes a lot more sense to say you can only eat it once, rather than you can eat it as many times as you want. It's not wrong; it's just a different thought model. A thought model that, incidentally, is the foundation of a very popular programming language.
One can say "Ok, but what if the postulates imply a contradiction? Surely that's bad, right?" Well... yes, but actually even here there's a lot of subtlety. Not for mathematicians, but for everyone else: see, the trick to defining logical systems that dodge Russell's Paradox is to have a cumulative hierarchy of universes, which is what a lot of dependently-typed proof systems use. But it turns out this is an enormous pain in the ass to work with for writing normal software, to the point that literally nobody does it, and instead we settle for simpler systems that are much more ergonomic to write in yet still, in practice, give a pretty strong (but not rock-solid!) guarantee that you haven't contradicted yourself.
Anyway, constructivism is a more grounded model compared to classical logic, in the sense that it actually computes results, and you can do the legacy thing by just saying "When the law of the excluded middle holds, XYZ."
More options
Context Copy link
More options
Context Copy link
I'm skeptical of math as a field and how Tao practices it. I think there's a chance he's an invention of his parents and academia and so secret shilling for OpenAI is right in his lane as far as attention seeking behavior goes. Take this with a grain of salt though, I don't want to slander him. He's certainly intelligent but yet none of his ideas have led to anything tangible and the praise heaped on hims just feels like it could be better allocated to practical problem solving, so it makes me a little suspicious that there's a confidence trick going on under the surface. Also, if there's anything to this, he wouldn't be the first, Von Neumann was suspicious too, for example his uncle mysteriously attributed to him physically impossible abilities that have been later debunked. Again, not saying these men aren't smart, I think they are extremely intelligent and what they do is difficult, but that labels like genius are political in nature and that these men and their entourages seem to do everything in their power to claim that label, behind just focusing on the best contributions to math and science they can make. To the extent that I think their contributions to humanity suffer as they chase fads and prove abstractions instead of undergoing less prestigious work to make material progress.
This is an important concern.
Tao has almost certainly been offered dump trucks full of cash to join quant funds. Perhaps if he made global derivatives arbitrage a few points more efficient he could save every American's retirement fund a few hundred bucks a year. Would that be a better contribution to humanity than whatever he's doing now?
More options
Context Copy link
There is tangible real world improvements that Tao's work has led to, like MRI scanners is brought up constantly as one of the most prominent and direct examples. It's true that pure math doesn't have many easy and obvious results like that, but that's true of a lot of science. Tons of important inventions are based off things that no one was expecting like microwave ovens (just a guy discovering candy melted in his pocket while working on radar equipment) or penicillin (accidently contaminated a petri dish while studying bacteria).
For every compound found in a gila monster that cures obesity, there's hundreds (thousands? Maybe tens or hundreds of thousands?) of other compounds that aren't found to do anything usable (at least not that we currently know of) but to find the former we have to go digging through the latter. We can make some educated guesses, but we don't know ahead of time too well what will be useful and what won't be. After all, that's why we do the work and the research, to find out.
ChatGPT indicates that maybe 10% of the improvement to MRIs from 2000 to 2020 was due to pure math, and Tao was maybe 10% of that. So that's 1%. But it says most of the improvement came from applied engineering using math as a tool.
But pure math gets way too much status compared to how likely it is to find something useful/interesting. Most of it is pretty obviously a waste of time. Quantitative science such as physics is a much more efficient way to search useful state space with math IMHO.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I don't really understand your overall point. To whatever extent Phelps' success was said to be thanks to wheaties, it's obvious the same can't be said for Tao for the simple reason that wheaties are from 1924 (predating Phelps) and Tao won the fields medal in 2006 (predating ChatGPT). It's obviously a logical contradiction.
The other thing is that we are at the point where LLMs are solving open problems with minimal or no involvement from humans:
Automated theorem proving across open Erdos problems. Each solved problem cost a few hundred dollars, which is hardly an unbelievably large number.
A problem relating to sumset combinatorics. Gowers is also a Fields medalist and seems pretty impressed with ChatGPT here. Reading the post, it does not seem that it takes a Fields medalist to do the prompting here: "the era where you could enjoy the thrill of having your name forever associated with a particular theorem or definition may well be close to its end." This also doesn't seem to have cost an inordinate amount of money, although there isn't a figure given.
A disproof of a conjecture related to the planar unit distance problem. I'll grant that it's not explicitly clear what the human prompting looked like, but there's no mathematician whose name is attached to this so I can't imagine it was something that only a select few could have done.
This is not an exhaustive list.
In other words, you don't have to be Tao to find new results with these tools. I am sure your response will be that nobody cares about these results, but unless you were predicting beforehand that we would get to this stage but no farther, it's hard to take this seriously.
Sure but that still doesn’t eliminate the problem exactly. It’s somewhat reminiscent of when Fermat’s Last Theorem was proven by Andrew Wiles. The early stage formulated results he came up with he remained confident about, until it took other mathematicians joining in to show him where he was off track. You don’t have to be a Tao to “find” these results per se, but how do you validate them?
It's frequently the case that it's easier to check a result for correctness than it is to generate the result in the first place. This is especially the case if the problem can be formalized in Lean.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Wait, Teortaxes is Dase? How did he manage to stop rageposting for his twitter persona?
He didn't stop lol.
I've read several posts by Teortaxes where he didn't say all his opps think Chinese people are bugmen, for your information.
He has posts on this site where he doesn't do that too tho. I may just be annoyed because he blocked my on twitter after accusing me of being a Maga guy after his flame out here.
Yep. It's him alright.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Re: Tao and the "precarious financial situation", I think that's just whining over "Waaah, my funding got cut and it's all Trump's fault!" If he really isn't being paid his stipend/salary, then he should kick the university financial department up the backside, not go on Substack to Orange Man Bad. But then again, I think Tao gets treated like a living god by some because of his undoubted mathematical genius.
Which just demonstrates why people go "If you're so smart, why ain't you rich?" He prefers (and maybe is only capable in) academia, which means he is going to be dependent on that kind of funding to make a living. Nobody is forcing him or playing 4-D chess to impoverish him so he has to be the spokesmodel for AI.
As for the rest of it, I have no dog in any fight concerning universities (should you go there, are the Big Names any better than empty signalling, etc.)
More options
Context Copy link
I’m confident Terrace Tao is pro-AI both because it funds him and he finds it interesting and potentially useful. That’s academia (usually).
OpenAI’s model did solve a long-standing Erdos problem (not in Lean, hand-checked by mathematicians, but still)
I think crypto is a good analogy. There is actual tech there, and many do indeed believe in it (me being among them). But the hype and noise was wildly disproportionate to what was realistic and done.
At this point, if you put your bet on anything other than "Slightly improved Bitcoin with privacy that was obviously intended originally but not known how to do at the time," you’re probably down 70+%, if not entirely liquidated. Entire narratives about "business on the blockchain!" were complete nonsense.
Fun fact: Sam Altman himself launched a crypto coin back in the heyday. It's down 96%.
I think crypto isn't such a good analogy. I never saw anyone get value out of crypto qua crypto. As an asset and an investment, yes, and occasionally as a way of paying for mildly shady or super-techy things, but in general the value proposition just never seems to have manifested to me.
Whereas I get massive value out of AI. For writing, for my hobby projects. My startup would be facing much larger headwinds without AI for coding and research. I think the hype is still kind of overdone, but only because the hype is so strong that only the immanent eschaton could live up to it and because it's not clear how much of a directly-related ecosystem there will be for third parties.
I have mixed feelings about AI; I have concerns about it being used to automate military decisions that should require human moral judgment (the traditional Terminator-style concern over computer command and control), and also the potential for deepfaking and manufacturing false content to mislead or manipulate. The latter has already been used in new and more sophisticated scams, and I worry about what a nation-state-level actor could do with that kind of power. Economic disruption is there as a genuine possibility, and that's difficult, but I'd prefer if people expressed that possibility directly as a livelihood threat rather than trying to launder the (genuinely sympathetic) concern into environmentalism or moral grandstanding about human creativity or interpretations of IP law in which AI training is assumed-illegal.
I've rarely actually heard someone say, "I don't like AI because doing my job without it gives me satisfaction and a good-paying job, and the introduction of AI into the workplace makes me feel like I'm losing the livelihood I prefer." Instead, I typically hear things like "AI was developed by stealing the intellectual property of hardworking people in order to enrich the billionaires and ELON MUSK and DONALD TRUMP," part of the large egregore of "all my enemies are evil rich fascists."
People would rather be angry than admit vulnerability. Our discussions over issues of social importance would be strikingly improved if people were willing to admit when their principles are self-serving -- which there's nothing wrong with, everyone deserves to advocate for themselves -- instead of trying to convert everything into an argument in which justice, law, the hand of God, and the long arc of history all militate against whoever you think is opposing your interests.
I don't agree with the environmental or land-use concerns for the most part, and it strikes me as degrowth corporate-hate and NIMBYism rather than principled objections. Energy use is not automatically immoral. I'm disappointed in the ways in which AI's demand for silicon is draining the consumer market of computer components and I worry about the impact on individual people's ability to control the means of technological production, but at least so far, this is offset to me by the increase in the ability to interface with computers using natural language.
The kind of generalized AI hate I see out there, online, occasionally in person, is hard for me to wrap my head around. I'm in the 10% of Americans who are more excited than concerned about AI. Generative AI has been great for me, in ways similar to what it's been for you. I enjoy using it. I get value out of it. I think AI slop memes are funny sometimes. I don't like when it's used to write personal messages or fill out marketing boilerplate copy, but I don't hate AI text as a general principle, especially if it's used to bolster and not replace human effort and creativity. And I dislike the invective and contempt that valid uses of AI generate in critics far, far more than I dislike the silliness or laziness of uses of AI that are in poor taste. That's the self-interested vulnerability of my own: I don't want a tool that has expanded my capability to become socially radioactive.
I don't know enough about AI to comment with any level of expertise on the research frontier. But I do have a skeptical prior towards the idea that this generation of AI will produce genuinely generalized AI that can meaningfully, affordably, and trustworthily replace human oversight. But we've gone farther with agentic AI use than I would have expected, so I might be wrong about that.
I agree with pretty much all of this.
More options
Context Copy link
More options
Context Copy link
Minor point, this seems to be the wrong way round. The asset aspect is the boring (if hyped) part, and the proposal to do financial transactions without a trusted party (which can easily coerced to block some transactions by the feds) was the innovation. Of course, this freedom to do transactions has mostly been used in darknet marketplaces and for ransomware, but that's humans for you.
I will grant you that the anarcho-libertarian utopia promised by the blockchain has not happened, though. 'crypto' is 99% get-rich-quick scams, and the 1% are probably mostly ransomware and the like, with 0.01% being nerds buying acid or donating to wikileaks. Legal crypto exchanges are very much centralized, and banking laws in the US are probably broad enough that the feds can jail you for decades if you put substantial amounts of your money through a mix or otherwise annoy them.
And gen-AI is definitely the bigger deal, sure. It might take six or eight orders of magnitudes more money to train a LLM than it takes to train an individual human, but my feeling is that if we assume that the tech will keep the current intelligence level and and simply improve on the execution, that is already enough to make the mean white collar worker obsolete. Heck, I have a PhD-level education and consider a future where I am reduced to wearing AR goggles and connecting cables to where some AI decides they should go while it takes care of the software tasks far more efficiently than I ever could distinctly possible.
This is not to say that the AI bubble bursting is not also possible. I mean, investors in the late 90s were not wrong about everything -- the internet did have an enormous effect on commerce. It was more the specifics which they were wrong about, like if pets.com would ever become profitable.
More options
Context Copy link
For complex tasks though, or for cutting down the massive overhead? I just think for most of the use cases I keep coming across, AI is nothing more than a fancy lawnmower that saves you time to cut the grass, but doesn’t do anything revolutionary for you.
That line gets fuzzy.
I can't talk work examples, but a hobbyist thing I've been crunching for the last two weeks is building a couple small educational robots.
That's not revolutionary, in the sense of completely breaking the field. It's something I've even done before at smaller scales: the first DIY educational robot kit I provided for a summer camp is almost a decade old now. But it's the sort of thing that's a massive time investment, especially when you're looking at a new microcontroller architecture or building something far from the standard line-follower or simple ESP32 websocket racer. Figuring out chip documentation, finding actual sane BoM materials instead of the wacky versions people go with at unit size 10k, managing errata, sanity-checking EMI, it's a nontrivial effort at even the smallest scale. On that side, AI's probably dropped it from a month of nights-and-weekends to a week or two, and probably made it better or surfaced information I would have missed otherwise.
(though even there, being able to actually find and translate information has encouraged me to go a lot broader than I did in 2018: there's been a few chips and targets I can genuinely evaluate five or ten options now, where before it'd just be a matter of finding anything not-EOL.)
The harder part is where I want to sell these things. People did that, pre-AI, don't get me wrong. But it was insurmountable to me, and probably insurmountable at my expected business scale. The actual build and development costs are trivial compared to the compliance costs, just figuring out the order of magnitude of the compliance costs meant hiring an expert, and worst of all, there's a lot of landmines I knew about even then which could invalidate a lot of your past efforts all at once, and others I didn't.
That's not, pointedly, revolutionary to the world. But it's revolutionary for my use case.
More options
Context Copy link
As Napoleon once said, quantity has a quality all its own.
My private project is a graphics thing for ricing. To get what I wanted, I would have had to become proficient in desktop compositing, OpenGL, wayland, and several disciplines around graphics and rendering. Then I would have had to write several thousand lines of fairly finicky boilerplate, including several false starts and bad assumptions.
If I were retired and had the time and the energy, I could do that. In practice, though, switching from 5% ideas 95% grind to 60% ideas 30% reading 10% grind means that it’s fun and I’m a good chunk of the way there after maybe three good evenings of work. Without AI that just wouldn’t have happened and it would go into the bin of ‘someday’.
For my startup, again, AI is not a superintelligence but it sirfaces good papers, explains the maths when I get stuck, implements diagnostics in minutes that would take me hours. It’s not like having a Nobel winner in my pocket, it’s like having a textbook that can talk to me and a bunch of PhD students on Speed. Very senior people in very serious organisations are using it for proof of concepts and your projects.
TLDR: no individual thing it does is truly revolutionary except maybe the maths from my perspective, but I find the ease and quality and speed with which it does it is revolutionary in aggregate.
That just proves the point though. That also holds true for most things in the industrial world. The gap for me stems not from it providing no value, but how it differentiates itself from everything else that achieves the same thing in its respective domain. There’s one of two categories the tech falls into:
AI making existing technologies easier to use and increase productivity.
AI inventing new tools, technologies, methods, routines and research.
The problem I have with so many people who love to talk up the AI ladder is they use 1 as a way to argue for 2. 1 has been the whole long read of technological and economic progress since humanity has existed. There’s nothing “new” about that. I’m glad in your case it’s lowered the barrier to entry for you, but I don’t see that as a strongly given “new inroad” for the tech itself.
I think that's a broadly artificial separation. In my opinion the vast majority of new tools / technologies / methods / routines / research come from some combination of:
I have observed AI doing (2) and it makes (1) and (3) considerably easier.
If my project works it will be an entirely new way of doing desktops, and I guess it was my idea not the AI's, which is maybe what you mean? But I got a lot of the techniques from another area and 90% of the design is the AI's suggestion and uses techniques I'd never heard of, so it's still more complicated. I'm quite happy for the top-level what to stay my job and leave the how to the machine, of course.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
The primary smell test is if a person promoting something is only using one brand. Does he only use chatgpt and never mention something else? Then that is fishy. Does he hop between models and providers than it is probably genuine excitement.
Speaking of covert LLM marketing I was recently invited to a meetup that specifically mentioned claude code vibe coding. There was no official sponsor yet they had rented a somewhat fancy venue and there were heaps of food. They were going to another city to do the same thing the day after. According to themselves they were just passionate about claude code.
Considering the boatload of money invested into these spaces it makes sense to market their products and traditional advertising probably gives less bang for the buck.
Typically, yes, but we're unironically talking 160+ IQ here. He certainly has negotiating power and some sense of dignity and taste. Anyway, yes, I freely concede my searches specifically for "OpenAI gives TT a bunch of money" were in vain. But they did turn up his new foundation, which comes remarkably close to confirming my original conspiracy theory right on the tin. He’s clearly salty about losing that government money (and rightly so), and is pivoting to asking for private sources.
More options
Context Copy link
I know little of the math / AI space but google is genuinely a player here in LLM math along with OpenAI
Has anyone ever seen Terrance speak of Gemini or use a Google LLM math product
I wouldn't read too much into this. Has anyone ever seen him use a theorem prover other than Lean? If anything, he's a lot more explicit about using the Lean "brand" than the OpenAI brand. Yet obviously he's not a sponsored shill for Big Lean (because there is no Big Lean).
What's clear is his organisation is pursuing private funding since the government axed his grants. I think it's likely OpenAI is among them given his appearance on their Twitter promo. Whether there's any exclusivity contract, who knows, I don't have a confident assessment.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I do not understand where you were trying to go with your digression about constructivism. Is this just supposed to be FUD, as in you hope non-technical readers will look at it and walk away with an understanding like "what Terry produces with his AI is not a real proof"? Because the reality is rather the opposite: any proof in a constructive system like Lean is a proof in a non-constructive system like ZFC, and certainly it is especially a proof in the unspecified system that is mathematical practice outside of a handful of particularly rigorous subfields.
I don’t think we’re referring to the same constructivism. I’m referring to this: https://en.wikipedia.org/wiki/Constructivism_(philosophy_of_mathematics)
More options
Context Copy link
Come on, that is not an honest reading of what I said.
In constructive systems, one proof—in the Curry-Howard sense—is not interchangeable with another. For example, you can sort in O(n^2) (or worse), which would be a perfectly valid thing to do in classical mathematics, but a big oopsie in formally-verified software. This is doubly-so in cases like cryptography, where almost none of the "incorrectness" of compromised crypto is incorrect at all in the classical mathematical sense. Further, as someone who has a fair bit of experience playing around with these systems in the past, theorem provers often induce you to write code in a way that is easy to make formal statements about, rather than in a way that actually runs well enough that anyone would want to use it.
My point is the software verification story—which is surely of high importance to businesses—is far more nuanced than "throw the clanker at Lean and let it churn". I think the bucket of cold water I’m tossing is completely justified (even as someone who does indeed buy the baseline premise—that LLMs will be a big boon to working with theorem progress).
Your general argument is correct, because proofs won't scale without efficient tactics and smart sub-lemmas, which can't be algorithmically verified.
But Cunningham's Law obligates me to point out that proofs are interchangeable, it's called proof irrelevance. A value is basically only considered a proof if it's type's type is
Prop, and once the proof is verified (i.e. value is type-checked), Lean can forget it and only remember that the theorem is proven (i.e. type is inhabited).Well, if we're going to be pedantic, proofs of kind
Propare irrelevant. I haven't followed this stuff in a while, but I'm pretty sure this opens a big can of worms, especially wrt erasure. Google's AI assures me this is all totally resolved and there's zero overhead, then links me to this thread which is not that old and smells of all the swamp I remember from back when I followed this stuff.More options
Context Copy link
More options
Context Copy link
Are they selling these things as proof of applicability to software verification? That seems silly, for many reasons. The domains share some language, but diverge a lot too.
The role of constructivity is a bit mysterious to me here. The headline results (at least the unit distance problem) were indeed constructive. On the other hand you can verify classical logic perfectly well by assuming, for example, a double negation oracle. Your programs won't run, of course, but they'll type check fine.
I mean, that was certainly my impression. Maybe I've self-selected into circles where that's the sort of thing people naturally care about, and incorrectly assumed this is how everyone is thinking.
Sure, if all you care about is Erdos problems, I guess much of what I'm saying is moot. But this whole "Mythos is finding all these vulnerabilities, the whole meta has shifted!" narrative sure leads me to the next part of the story of "Ok, so how do we actually build stuff correctly in the presence of tools like Mythos?" Because the mythos story, like the classical theorem prover story, is also basically invulnerable to the primary problem of LLMs, hallucination. If the model hallucinates a vulnerability that doesn't work, just throw it away and try again! (Or, in the case of low-human capital, spam the issue tracker with bogus vulnerability reports. But I digress).
I think you made up a thing that people are doing and then wrote a post about how it's dumb that they are doing that.
I mean, yes?
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I still can't make sense of what you are trying to say then. What is the sense in which you claim proofs to be "interchangeable" in classical systems, why do you believe this to be a desirable property, and how do you contend with the fact that constructive proofs are classical proofs?
There may be some HoTT-like sense in which two proofs in a given system are not equivalent (because you introduced some equivalence relation on proofs that does not relate them). If you have a sound embedding from proofs in this system to proofs in another system, and the other system comes with its own equivalence relation on proofs, the images of your two proofs under the embedding might still be equivalent in the other system. Is this not just what would happen here, if you assert that all classical proofs of a given sentence are equivalent (under an equivalence relation you picked) but not all constructive ones are (under an equivalence relation you picked)?
You will have to elaborate on this statement. What do you take to be an instance of compromised crypto that is not "incorrect in the classical mathematical sense"?
Well, in the classical mathematical sense, crypto doesn’t work at all: just factor the composite number. The entire premise relies on the relatively ill-specified (by mathematical standards) notion of relative computational cost disparity.
What I mean is when I sit down to write something in a theorem prover, I speak in terms of Peano nats and inductive lists. When I write software anyone would ever want to actually use, I use machine ints and arrays. There is definitely a sense in which I’m doing the same thing in both cases, but nailing this down precisely is… well, non-trivial. (aka, a royal pain in the ass). Like, the discrepancy here is show-stoppingly problematic.
EDIT: basically, the computation is relevant, and more specifically the speed of computation is also relevant, both in terms of practical usability and outright security in the case of crypto. In the classical world, there is no notion of computational relevance at all -- in fact, you outright end up with these counterintuitive weirdo theorems like Zorn's Lemma. The counterintuitiveness depends entirely on this conspicuous lack of computational awareness in the model: as soon as you put that ingredient back in, you're in the construcivist world, and silly results like Zorn's Lemma don't hold anymore unless you postulate them. My contention is the class of problems for which one can say "I don't care about which proof, just that one exists" is relatively narrow and of little practical relevance (in the sense that one can get an incomputable proof, which is, in a fundamental technical sense, useless), while the class of problems about which one does care which proof, in the Curry-Howard sense, is large and of enormous practical consequence. I want to argue that enthusiasm over the former should not translate to enthusiasm about the latter.
More options
Context Copy link
I think he means it in the sense that "a nice man on the phone told me he was from BitPanda and asked me to read my password to him so he could check if it was secure" or "SBF stole all my money" can't be detected by enforcing code correctness. All of the badness has happened outside the code, everything inside the code is a perfectly valid transaction.
But this general argument (ChatGPT can claim something in English, then formally prove something completely different) still applies if the proofs are classical.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Who is saying anything about AI assisted software verification via Lean? I've literally never heard of anyone suggesting that we should do this.
Terrence Tao literally talks about this in his Lex Fridman interview:
https://youtube.com/watch?v=hh4cjZOddQA
No, he's talking about theorem proving, not formal software verification.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Yeah, I'm not sure what his point about "the proof isn't canonical" is all about. A proof is a proof, and the underlying statement is just as valid regardless. The fact that there's no canonical isomorphism between a finaite dimensional vector space and its dual doesn't mean the two spaces aren't isomorphic or any such isomorphism is less "real".
Part of the misunderstanding is the word "proof" has different flavour in classical mathematics vs theorem prover math, which is why the word is rarely used in this world except when appealing to legacy mathematicians. The native vocabulary is "3 is a value of type Nat" or "Refl is a value of type 0 + 0 = 0". A legacy mathematician would use the word "proof" only to describe the latter, and think it odd to use the word for the former. But internal to the formal logical framework, these are same notion, and so they should have the same word.
Neither of these would be considered a "proof" by a mathematician. Those are just statements of fact. A mathematical proof is a logical argument of deduction that shows that some statement must be true if some set of premises are true. Statements of fact are used in proofs, but a single statement of fact wouldn't actually constitute a proof.
I think you've never spent any time with a formal theorem prover lol. You're conflating the definition of
+with the proofRefl : 0 + 0 = 0.Mathematicians can say whatever they want, but the story here is about mechanically-verified formal verification, and this is how formal verification works, whether legacy mathematicians think this pedantically or not.
(emphasis added)
So it seems like you're walking back the claim you made about what a (legacy) mathematician would say. Again, a legacy mathematician or any mathematician wouldn't describe that as a "proof."
An appeal to a definition is a proof ("By the definition of XYZ, we conclude..."). A definition itself is not a proof ("Let xyz be defined as ...").
The confusion is coming from the fact that
Refl : 0 + 0 = 0is the syntax of an appeal to the definition of plus. It is not the syntax of defining the+operation.Look, I'm not going to run around in circles all day about this. I'm pointing out that mathematicians are more sloppy in their vocabulary than the theorem prover, and that there are subtle differences over how terminology is used. I do not care to bikeshed endlessly over this. It's obviously true to anyone with any experience in these fields.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link