Shirayuki2
new account of Shirayuki, lost old password
User ID: 4180
Then why is it that so many students who spend hundreds of hours studying a foreign language in school come out the other end not knowing how to speak them?
Because in monolingual countries with a large population, neither the student not the teacher cares enough to have the student come out with a strong grasp of the foreign language on the other end. In countries where getting good at English or another lingua franca is a practical necessity, you better believe a lot of students are coming out with functional language skills despite many of these students having a fraction of the investment rich Anglo countries do.
Most kids have the capacity to learn a language really easily, because humans are designed to learn to talk. They don't have the capacity to learn two
From "Language is Culture" by Spandrell:
This doesn't make any sense. While obviously there are physical limits in the extreme, there is no evidence that language acquisition is bottlenecked by storage space of all things - if you have a GeForce 256 and a 100TB SSD obviously storage is not going to be the bottleneck in your system.
As Pigeon says bilingualism is very common globally - the fact that many children don't learn multiple languages comes from a lack of incentives rather than a lack of ability. Many people across South America, continental Europe, Africa, and ex-CJK Asia will at least be conversant in a global lingua franca and a native language out of necessity: e.g the Nordic countries and the Dutch are almost all perfectly bilingual English speakers.
From "The Myth of 'They Weren't Ever Taught…'" by Education Realist:
And from "The Numbers Speak: Foreign Language Requirements Are a Waste of Time and Money" by Bryan Caplan
These are indictments against the requirement to learn languages in school, which I agree is useless for unmotivated students, but really this is a general argument against teaching anything in school above basic numeracy and literacy. I broadly agree with Caplan's Case Against Education thesis, but these posts do not support your bailey that secondary language acquisition is not possible or detrimental.
The motte here is that the opportunity costs of language learning are high, which I think is probably correct for English speakers even as a hobbyist polyglot myself, but in the long run everyone is dead anyways. There are much worse and unproductive things that most people do with their lives.
Shirayuki2
7d ago
Houellebecq was writing about this thirty years ago. The sexual revolution has indeed always drawn sexless men into the extension of the domain of struggle.
So I'm left wondering what sort of wealth tax could be implemented, in a way that doesn't destroy too much value while still being meaningful enough to appease the envious
It's an interesting problem, and I'm convinced this is going to be one of the largest issues facing society in the years to come, but at least I cannot think of any easy solution.
On the first hand, I broadly believe that wealth inequality is not a problem in and of itself - it makes no particular difference that someone else has much more than I do, as long as I have what I want and need myself. To an extent where it is a problem, in that wealth allows for the concentration of power, I've always thought that it was merely a symptom of the immutable fact that increasing technological progress allows for the increased centralization of power. If there was someone capable of expropriating the wealth of the billionaires, it merely means that that someone else has expropriated their power - the government can exert much more control than any private capitalist ever could.
On the second hand, empirically the median person really does not like experiencing wealth inequality, especially in the modern social-media driven age of envy. It's easy enough to say that if everyone was an enlightened buddha, then wealth inequality would not be a problem, but it's much like saying that resource allocation would be easy if everyone simply got along - simply impossible in practice. If it were possible to burn value in order to satisfy people's lizard hindbrains and improve social cohesion, I'm becoming more convinced that this would be a good thing, even if it were not optimal in terms of resource allocation.
On the third hand, it really does not seem like there's any level of reduced inequality that would actually satisfy the median person without ruinous consequences. Complaining about billionaires and trillionaires is a useful Schelling point, but in practice I do not see that "mere" centimillionaires worth 999 million are any less targets of jealousy - in practice most people are just envious of anyone with more than them, while believing that they're all fully deserving of what they themselves have. As others write below, there are many countries which already implement wealth taxes and/or have lower Gini indices than the US, but it doesn't seem to have meaningfully reduced the inherent envy that people feel towards those with more than them at all.
As a response to @magicalkittycat below as well:
An administration official told Axios they do not view other models as national security threats because they do not surpass the bar that Mythos set.
Anything at Mythos level or above would need to go through the administration to ensure the government's national security apparatus is hardened enough, the official added.
It seems that the USG is at least nominally interested in regulating models from the other H-bomb teams as well, and one of the claimed reasons that Anthropic is being targeted is because they're considered to be the ones with the strongest model worthy of regulation. Source.
Perhaps you don't trust the administration when they say this - it's hard to know what the second-hand effects of these expert controls are going to be, and this could certainly have been handled better by both parties. Still, if you're concerned about doom and would prefer increased regulation to try and avert doom, this seems to be at least a potential step towards something better.
In an ideal world we would have a council of LKY clones making maximally technocratic decisions, but we live in the world that we live in and nobody else is going to be stepping in.
Statement on the US government directive to suspend access to Fable 5 and Mythos 5.
A few immediate thoughts:
For one strain of doomer, this is a good thing: the USG showing willingness to directly intervene and cut down the addressable market for capability-pushing models reduces economic incentives to push the frontier at all costs and slows down the race dynamic. Correspondingly, it's going to be a nightmare for the markets: the AI boom is driven by the idea that frontier models are going to be replacing a significant chunk of global labor, and obviously now this is significantly less likely now that frontier models are going to be stuck in Uncle Sam's basement.
For another strain of doomer, this is a horrible thing: really this was fairly clear even with the DOD conflict, but now control of AI is now very firmly a direct White House concern - the fate of the planet and/or the universe is in the end going to be decided by DC bureaucrats and not SF tech nerds.
Companies, especially non-US companies, are going to be rushing to the doors to move their AI workflows off closed-source and onto self-hosted models, another blow for the AI market thesis - the business continuity risks of Anthropic's demonstrated willingness to silently cripple models and USG's demonstrated willingness to now arbitrarily cut off access are simply going to be too much.
Shirayuki2
19d ago
Money is an abstraction over resource allocation in this context - I agree that production is not zero-sum, but many of the inputs to production are obviously rivalrous; land, power, raw resources, construction workers and chip fab allocation are all finite and rivalrous to name a few.
If you're just trying to be edgy about lack of state capacity being why shit doesn't get built then I agree, in many cases resource allocation is not the limiting factor as to why no progress is made, but it's farcial to claim that resource allocation isn't the limiting factor for anything that might directly improve people's lives.
They are being built, just have not yet finished being built. and no one besides investors are being asked to give them anything
Money is fungible. To a first approximation, if a trillion dollars is going into trying to build data centers (and failing, in many respects) that's a enormous chunk of resources that isn't going into fusion research, hiring, fixing bridges, what have you - things that could directly and meaningfully improve people's lives.
FWIW I'm optimistic about AI as a normal productivity-improving technology for improving people's lives, but this is explicitly not what the frontier American labs are aiming for, and it seems extremely difficult to justify the trillion dollar investments in a AI-as-normal-technology world where tokens are inevitably commoditized and primarily inferenced locally.
Should we squander it?
I guess I'm just feeling a disconnect from your posts. If you want to be a hard-nosed historical realist, OK: in the same way that the Treaty of Versailles and the Great Depression made WW2 inevitable and Hitler was merely in the right place at the right time, I agree that race dynamics ensure in no realistic world does a bilateral pause happen, and that as soon as Attention Is All You Need was published realistically no force on the planet could have stopped the AI race from accelerating - Altman and Amodei were merely in the right places at the right times to ride the wave.
But then you go on about "how can you not be excited" and "[bringing] about a new age of American prosperity" and I'm just confused about how you can think we're being forced into a world where if anyone builds it everyone (probably) dies and think this is a good thing. At best, if by some miracle alignment is inherent, you're under the heel of God-Emperor AI Frontier Lab Leader or God-Emperor Democratic/Republican President (adjust for your outgroup party) for all time and at worst all America's done is won the "created the Torment Nexus first" award.
At the end of the day "American" and "Chinese" are merely constructs of society as we know it and not immutable laws of reality - I fail to see how "superhuman general intelligence" and "liberal democratic All-American values as we know them" or "superhuman general intelligence" and "socialist values with Chinese characteristics as we know them" could meaningfully co-exist, so what benefit does the median person really get from ultimate, unaccountable power being instantiated 2000 vs 6000 miles away from them?
The chance of it killing us all is coming either way
I generally agree, in the sense that "WW2 was inevitable as a result of complex geopolitical factors post-WW1", that "if the fate of human civilization is to create the Torment Nexus, then the Torment Nexus will inevitably end up being created", but obviously that didn't clear Hitler's name and being on the vanguard of potentially creating the Torment Nexus is simply not going to make AI labs popular.
I mean of course they haven't because those dollars haven't even been meaningfully spent yet because we haven't built the datacenters
I mean, is that supposed to make it a more appealing thesis for the average person? "You should give us a trillion dollars to build out datacentres that aren't being built, and maybe if you give us enough and we finally succeed you'll be lucky to get something out of it and avoid the permanent underclass or being drone striked by Skynet" is not exactly stirring rhetoric. I'm not saying that AI is useless or that it couldn't theoretically generate a lot of consumer surplus, but that's clearly not what the public messaging on the topic in the West is pushing.
The median person who hates AI thinks a literal bottle of water is obliterated from the universe every time you query chat gpt for anything
While it's true that populist anti-AI arguments are not very good and mostly arguments-as-soldiers, that's distinct from not having good reasons to act that way in the first place. There's a dedicated core of people who hate phones, cars or fracking, but most people just enjoy the benefits to their lives from those technologies and move on with their lives; there isn't really such consumer surplus and many externalities from the current state of AI, so it's no wonder why people start turning to populists and arguments as soldiers.
I think there's a fair chance this whole thing ends up with all of us dead
as well as a fair chance that progress peters
Are you really surprised why a technology where its proponents and developers are saying that it has a solid chance of killing everyone and/or consigning the median person into permanent unemployment at best, or creating a titanic financial bubble that's going to bring everyone's retirement with it if it doesn't pay off at worst, is extremely unpopular?
As a software engineer I enjoy using coding agents and I do think with demographics and fertility being what they are, the only real choice as a society is leaning into AI and automation as much as possible even with the commensurate risks, but frankly the median person has really not seen proportional consumer surplus from the trillions invested into post GPT-4 or so LLM development, and in return all they get is dealing with a tsunami of online slop, vibe-coded software that gets worse even as engineers boast about how productive they are, and higher prices for power and RAM.
At least in China they have the message discipline to tell people that AI is going to be used to improve people's quality of life, but all SF can do is jerk off about the AI-induced permanent underclass happening any month now and how dangerous AI is is going to be; it's obvious why the median person hates the AI build-out.
Shirayuki2
21d ago
While I agree with the premise that you hardly need to work 40 hours a week for 40 years to maintain at least equal and in most respects significantly superior material conditions than someone from the 1920s, it's impossible to get a 1920's level of existence via welfare or FIRE - living a 1920's life alongside friends, family and potential romantic partners in 1920 is significantly different to voluntarily choosing such a life while all your peers are living in 2026.
In that respect it's hardly the fault of the system, capitalism, or wealth inequality though, but simply the natural consequences of choosing to live like an oddball; at the end of the day most people complain, but their revealed preference is that they'd rather work more to consume more and fit in.
Shirayuki2
22d ago
While I align with the idea that in the limit the only real possible outcomes of superhuman general AI are doom or utopic abundance, I agree that the third point is proving too much. Possibility of AI being a bubble aside, even granting that AI lives up to the hype and drives unprecedented productivity, it seems extraordinarily difficult to determine who actually ends up capturing the value from that productivity.
Nvidia is a software company in a trenchcoat vulnerable to having their moat and margins forded by sufficiently good SWE AI, FAAMG still derive a supermajority of their revenue from vulnerable middle-class consumers directly and indirectly, and the American frontier labs are potentially vulnerable to having their margins on inference undercut by Chinese lab competition and tokens turning into a commodity.
Personally, my thesis is simply to be largely all in on global total market indices - reasonably diversified against a potential bubble, while insofar as property rights are preserved and AI creates unprecedented value, it gives me the best chance to invest in the someone primed to capture that value, and presumably even the "modest" returns on an index fund are going to be good enough if AI does end up being that transformative.
Of course, in the more exotic doom / utopia scenarios one's investment mix is not going to matter very much either way; betting on the rapture has never been a winning strategy.
Realistically, I think the only solutions that work at scale are technological advancements that obsolete the need to exploit the commons.
There's literally never been an instance where people "just" decide to collectively stop defecting, rather the payoff matrix gets changed by technological advances and it becomes irrational to defect.
Limiting CFC's went so well in large part because it ended up being pretty easy to transition from CFC's to HFC's, and peak fossil fuel is approaching not because of the environmentalist or degrowth lobbies but because the costs of renewables are falling precipitously. On the other hand, trying to prevent antibiotic resistance has pretty much gone nowhere because antibiotics are so hard to replace - I have much more faith that we'll get better ways to kill bacteria before any progress is made trying to coordinate usage at a global scale.
I think it just depends on how adversarial the judging ends up being rather than saying anything about capabilities.
If the judges have to stay within guardrails then 3.5 could probably win the longbet, but if they're allowed to exploit jailbreaks or known LLM failure cases, then nothing short of ASI is going to pass the test.
Shirayuki2
1mo ago
This is much more maximalist than even the AI 2027 crowd or the actual frontier labs themselves but I respect providing specific events and an end date on the prediction.
My own expectation is that none of this happens by the end of 2027 except a tranche of models that are notably better at RLVR'd tasks and the Copilot button (shorthand for AI getting more integrated into white-collar workflows). Let's see what happens in 18 months.
Shirayuki2
1mo ago
their R&D spend is worth it, even if crazy
Well, burning other people's money to try and build a moat is obviously worth it for the frontier labs. It's yet to be seen whether that spending will be worth it in the sense of paying off investors or building the labs a durable lead, or whether the models will end up commoditized and value accruing elsewhere in the stack.
Shirayuki2
1mo ago
While inference having high margins is true, there are two things to keep in mind here:
Amodei has never said that models are actually profitable on a per-model basis, only that they hypothetically could be. While this might be true, there are trillions of dollars on the line to insinuate that it's true, and personally I wouldn't trust any rumors about financials from a private company who can massage them however they please.
Spending the GDP of a small country on R&D on the promise of getting a commanding lead is why OpenAI and Anthropic have trillion dollar valuations to begin with. There's no such thing as a frontier lab who can cut their exorbitant capex and coast on the margins from inference, as that's a one way road to getting cut-throat commoditized.
I feel like a lot of people in these replies are talking past each other.
My 2 cents:
Are LLM's useful tooling for finding vulnerabilities for security researchers?
Yes, I think this is undeniable at this point; LLM's are exceptional at uncovering software flaws, bugs, and vulnerabilities, and are going to significantly change how cybersecurity is practiced, as can already been seen by how vulnerability disclosures have recently quantitatively spiked like crazy.
Is Mythos better than the other available models at finding and exploiting vulnerabilities?
Yes, Mythos really being a stronger model for cybersecurity applications is almost certainly the case: this XBOW report is a good read on its capabilities.
Is Mythos a super-hacker that's going to break cybersecurity for good?
No, this seems unlikely and driven by good marketing from Anthropic and online hype. Mythos isn't making the Move 37 for cybersecurity or discovering vulnerabilities beyond human comprehension, it's just an iterative improvement over the current tooling combined with a lot more compute and attention suddenly being used to uncover security vulnerabilities. I suspect that the same amount of compute, security researcher attention and buy-in for Project Glasswing applied to the previous generation of frontier models would have uncovered the majority of security issues that Mythos did.
It's also worth noting that there are apparently 11 Curl CVE's in the current release cycle, where the new CVE's did not use Mythos, which seems to disprove the idea that Mythos was not all that effective on Curl because it was uniquely hardened or secure.
Should LLM's being good at vulnerability discovery and theorem proving be an update on LLM's eventually reaching AGI?
YMMV, but to me, the recent headline mathematics and cybersecurity achievements haven't really changed my view that AGI emerging from LLM's seems unlikely. From an outsider's perspective, most of the recent gains in model performance look to have come from RLVR on coding, math and cyber. While very effective at improving performance on those tasks, it seems that RL has largely failed to further generalize intelligence beyond the specific RL'd areas, and if you look at SimpleBench or the AI RP community, seemed to have regressed performance in other areas of intelligence.
I think it's telling that all of the achievements of LLM's being held up over the past ~18 months (METR eval, CCC compiler, theorem proving, cybersecurity), while extremely powerful and which make me bullish on the utility of LLM's, are all tasks limited by requiring an external oracle for verification, and where there's no penalty for failing during intermediate steps. I personally think it's quite likely that LLM's eventually become superhuman at proving theorems and exploiting vulnerabilities given sufficient compute, but still cannot manage a restaurant, write an interesting book or autonomously maintain a software project.
Shirayuki2
1mo ago
What are some unusual items you've got on your bucket list?
Japan
Losing the war and being occupied by the allies was probably the best thing that could have happened to Japan's economy, as opposed to dragging them down.
The American occupation broke the Japanese military-industrial complex and forced them into exports and free markets, funneled them large sums of capital via Korean war procurement, American market access and technology transfers, and extended Japan the American security umbrella. The Japanese economic miracle wouldn't have been possible without losing the war and getting dragged into a modern economy by force.
India
While they never went full communist like China, there's a good argument that India could be decades ahead of where they are now without Nehru and Gandhi fucking around with poorly implemented socialism. The over-emphasis on heavy industry, licensing and central planning, failing to implement any real land reforms, and essentially being closed off to trade prior to 1991 were practically the completely opposite conditions as to what made the four Asian tigers so successful.
While I agree that India has different issues to East Asia, having a 50 year disadvantage on Japan and a 10 year disadvantage on China in liberalization did them no favors either.
Shirayuki2
1mo ago
Just wanted to say thanks for posting this.
I've been in a bit of a depressive spiral for the last few days and this post really helped.
Shirayuki2
2mo ago
My understanding of calvinism vs middle knowledge doctrines was that the main difference is that Calvinism treats free will as compatibilist while Molinism treats free will as libertarian.
Under either doctrine God must still have foreknowledge of children being born with genetic defects that kill them a few days out of the womb and foreknowledge of those destined to die during natural disasters; otherwise how could He be omniscient?
Under theism, you can consistently make choices that go against your self-interest and sacrifice both self-interest and prudence for the sake of morality
Sure, but the point I'm trying to make here is that moral epistemology becomes incoherent, if you abrogate omnibenevolence as understood by human moral intuition. If the meaningless suffering that pervades the world is all part of His plan, then there is no reason to morally privilege doing anything over, say, kicking as many puppies as possible. For all I know that kid drowning to death over there is all just part of the plan, and He instead wants me to kick the shit out of more puppies instead of saving the kid.
Maybe skeptical theists can bite that bullet, but it's not a bullet that I feel capable of accepting myself.
Shirayuki2
2mo ago
While I think Buddhism in general has useful lessons in reducing suffering even for an atheist, I don't understand at all how western, secular Buddhism can be logically sound without being a suicide cult.
If you accept dukkha, samudaya and nirodha, but you don't believe in the Right View, in the tenets of karma or rebirth, it seems immediately clear that that instead of the Eighfold Path, the much more efficient method to achieving nirodha is the Singlefold Path of a bullet into your skull.
Shirayuki2
2mo ago
I have to admit I find theodicies appealing to skeptical theism abhorrent, and personally I've never managed to get over the problem of evil.
Why should man do anything good, why should man do anything at all, if human moral intuition is meaningless and the most profane acts can be justified under an unfalsifiable appeal to the greater good?
Rats and ants may not understand the motions of men, but neither should rats and ants have any reason to worship men except for the last argument of kings, the threat of pure brute force and violence.
If God's vision is that children should die screaming in unimaginable pain and that the Ichneumonidae should eat caterpillars from the inside out, then frankly in the footsteps of Ivan Karamazov, I don't want anything offered to me by such a god.
Amusing to me that magicalkittycat got mass downvoted without substantial rebuttal to his post for expressing what I thought were very reasonable points, but also got an AAQC on the very same post. Not sure what so many people found so objectionable with the original post.
More options
Context Copy link