EverythingIsFine
Well, is eventually fine
I know what you're here for. What's his bias? Politically I at least like to think of myself as a true moderate, maybe (in US context) slightly naturally right-leaning but currently politically left-leaning if I had to be more specific.
User ID: 1043
There are basically 3-5 types of players of note. The government, large stack tech providers, and data brokers are the most distinct and relevant ones.
The government is extremely capable but also doesn’t usually bother to assemble its data into a full-you, longitudinal picture unless it’s motivated to do so. Theoretically that requires a warrant or a high degree of suspicion but in practice it just requires a casual interest. I think regular citizens worry far too much about this and powerful citizens worry far too little about it.
There are only about 3 players in tech with large “stacks”. Google, Meta, and Amazon/AWS. Second tier players in terms of exposure or will to track include Apple, ByteDance, and Microsoft. Any other tech company relevant for an American only matters insofar as they integrate their stuff with the final group…
The “data brokers”. These guys assemble pictures of you based on what dregs they can buy from bigger players, smaller but more comprehensive deals with single or more focused services, and occasionally supplement with data leaks even if such is technically illegal I’m pretty sure they still do.
It’s important to keep these 4 groups distinguished (there’s a major gap between the top tier of tech and the second tier). The answers and usage of the data differ a lot. To some extent the top tier hold back from their full theoretical power.
I will add that there is probably a fifth group of relevance: ISPs and cell providers. These groups are theoretically high exposure but held back due to regulation or fear of lawsuits. The government teams up with them again in cases of suspicion but otherwise doesn’t usually bother. (Banks might count as a sixth group but AFAIK they are super regulated about what they do so don’t matter)
I’d say that the exact words and recordings usually aren’t a major worry. It receives too much attention if you ask me. Your location is far from granular but big picture is likely very knowable even by smaller players. The data brokers are a bit inconsistent but potentially the biggest store of info and also the least regulated. However that inconsistency also works somewhat in your “favor” as the knowledge they get is by nature very inconsistent. You’d be surprised at how hoarding some companies are about their own data and how reluctant the biggest players are to share the Crown Jewels even if they only sorta use it themselves. Your web activity is pretty patchy because the tech evolves so fast and there’s a major wax and waning of exposure. Sometimes they can track a ton and sometimes the noise is strong and it’s hard to assemble patches of data with reliability.
And again each of the 4 nongovernmental groups get different slices of the data so unless you’re asking specifically about the top tier it really depends.
EverythingIsFine
Well, is eventually fine
23hr ago
I also think there’s something to be said for how large male-dominated orgs have chosen a decision structure or maybe also a leadership structure that suits their strengths. I don’t think it makes sense to make this out to be more powerful than it is, but I think as you say even if women make equally good or even better (as I think some research suggests) decisions, time is money, faster can be better, and sometimes forcefully imposing decisions on others can also be more effective than we give it credit for. It does make me wonder is sociologists could invent a managerial structure that improves performance across several axes. However I think research on this also attracts hucksters and bad science, so it’s hard to tell a legit management consultant (assuming they exist) from a bad one.
EverythingIsFine
Well, is eventually fine
1d ago
Just approximating, I think it's about 30 dollars per taxpayer per year, (lazily 4.9 billion in 2024 / 161 million tax returns), for 20 million lives supported. A decent chunk less if you adjust for progressive taxation, though I'm too tired to actually estimate it. Just as a thought experiment scaling it up, would we burden every American taxpayer with 1-3,000 dollars per year to save 2 billion people? Maybe that's on the border, I think I'd lean yes. I don't think by itself PEPFAR counts too much as a slippery slope or anything because PEPFAR is kind of unique in terms of the cost efficiency or opportunity, there are no other PEPFARs. Disclaimer: not an EA type, don't know the nitty gritty, and happy to have someone correct me on the numbers if wrong.
Turok: the Epstein saga is materially true in a broader sense. Leave the conspiracy aside for a moment. You only have to look as far as his first brush with the law over the issue, where he leverages connections and money to get off with a slap on the wrist and the government not looking too closely. Even if the public is fuzzy on the details they more or less have the right general direction here in terms of rich shadowy men with connections getting away with stuff. Also: shocker alert, what you call the "online right" is not entirely made up of dishonest hacks who only care about the returns. Nope, they are by and large true believers. They don't give a fig about the actual long-term second-order effects. It's not some big plan that you're bursting their bubble about. There's no Santa Claus, here, in the first place, so pointing out he's not real doesn't do anything. In that light, I find your comment to be pretty pointless, and that's coming from me who kind of agrees about the status quo being unreasonably sticky *taps username, this time unironically and that generic replacement figures obviously quite often come from a similar statistical ideological and demographic distributions as the prior ones.
In fact I also happen to think that a large proportion of politicians are also either true believers or so ideologically captured as to make little difference, because of the pipeline that produces them, but most people seem to disagree and consider them almost all pure ambitious sociopaths with agnostic or apathetic personal politics, with a handful of outright corrupt ones on the side.
EverythingIsFine
Well, is eventually fine
4d ago
Might depend on the program. I was next door in stats but I knew a few Econ-Math or Econ-Stats double majors and the first semester Econ class was the most notorious on all of campus.
EverythingIsFine
Well, is eventually fine
4d ago
That’s fair to an extent. I think it hews much more closely to the history of the West Bank though. Also, how much has Israel tried to do anything real with the PA in recent decades, let’s be honest, not much at all. My understanding of the timeline is the nation-building was decent for the first five years or so but the Second Intifada, Camp David failure, and re-occupation in 2002ish wiped it almost all out, both trust and infrastructure, to fitfully restart a bit again for a few years, until Bibi 2.0 around 2009. Basically as soon as he showed up it went into a permanent stall/holding pattern at best, and Bibi’s preference was deliberately for a weak PA, so if anything state unofficial policy has been to undermine the PA where possible. That’s been the status ever since, for 15 years or so now. I should also note that the few years immediately before 10/7, this was especially noticeable (eg the PA was ignored in the Abraham accords). For fairness we should note 2009 is also when Abbas began clinging to power undemocratically.
The Gaza situation is a bit harder to parse. We follow a similar trajectory but with more radicalism and less autonomy and more violence on both sides (not equipped to discern scale but I think in this time some assassinations took place). Until the increasing violence, withdrawal, and 2006 elections with Hamas getting a plurality followed by a swift 2007 civil war overthrow. I think with respect to the analogy, for Gaza the clock on the analogy basically restarts there: a failed and violent state with religious extremist terrorists in charge, a total war of annihilation, occupation, all the things I compared.
So for West Bank you’d be fairly accurate in saying nation-building was tried (and the relative stability of West Bank is probably owed to this!) but for Gaza I think the Israelis need to seriously consider a similar game plan as the US.
EverythingIsFine
Well, is eventually fine
4d ago
I should have been more clear, yeah. But the thought remains, because as I mentioned the normal left (which even Vance distinguishes from the far left) are still mostly fans of the civil religion stuff. Other parts of the left aren’t as loudly America Sucks and in fact disagree. So I think he’s exaggerating the trend.
To attempt being more specific, if we use the Pew Political Typology groups, as of 2021 the “Progressive Left” is 12% of Dem and Lean Dem people, and though there isn’t an exact question on patriotism, there is this perhaps-proxy: “there are other countries better than the US”. It might interest you to know that there’s a huge chasm between them and the rest of the main Dem alliance. They respond 75% yes. “Outsider Left” (16%) also land at 63%. Here’s the catch, every other group, Left or Right, is at under 25% on this same question including Establishment Liberals and Democratic Mainstays.
If we’re talking immigration, as of then the “Democratic Mainstays” (28% of coalition) were notable doubters on a few measures. It depends heavily how you slice the adjacent questions though, for example “America’s openness to people from around the world is essential to who we are as a nation” has a sharp divide (70 and up vs 30 and below) but the Stressed Sideliners join an actual defection from the Ambivalent Right on the upper end.
EverythingIsFine
Well, is eventually fine
4d ago
This particular facility is better known for several rape settlements (about guards), a beating settlement, and holding El Chapo for a bit. I don’t get the vibe that it’s the kind of jail, due to its nature as higher security, where gangs have the run of it… not to say it isn’t plagued by typical jail management stuff. I looked a bit into the history of the place. There’s a few cases where a guard smuggled in cell phones, various drugs, and lots of other contraband, but one was for two people and was a money making scheme. The other was three guards and the inmates had local gang connections (they are after all criminals from the area in many cases) however the guards didn’t. By all accounts the place was miserable - worse than Rikers said one inmate, with a mountain of lurid corroboration. El Chapo himself allegedly had a mental breakdown after staying there for a just few months. All this combines to me to suggest the normal official outcome is more likely.
Now you’ll never hear me call it impossible. It’s plausible beyond a superficial level. But far from likely. Not likely enough IMO that treating it as a worst case scenario is logical to do. For the guards to escalate to murder of a high profile suspect like that a noticeable amount of money would have to change hands and the feds are pretty good at money tracking, for whatever else they sometimes lack.
EverythingIsFine
Well, is eventually fine
4d ago
I mean you’re citing the maximalist case, that I don’t deny happens occasionally, but sure. What I mean is that obviously bonds are relative. Bonds with whom? Is the bond of a neighbor with their longtime neighbor diminished because they have a third new neighbor? Not really (that’s how I initially parsed it). If we’re talking some kind of ambient social identity or affiliation, that’s more culture than sociality.
And I think there’s an excellent argument to be made that mere locality is a weaker bond than it used to be, with the decline of community institutions and our new media age, independent of who exactly is moving in. Plus, although I totally get where you’re coming from about ‘oh we might have less to chat about’ I think that’s not necessarily the case. If both newcomer and old timer have an interest and/or motivation to connect, it will still happen! Maybe you chat about football and they chat about soccer, to mutual interest, your Christmas invite is responded to with an Eid invite, etc. No need to be rosy here - maybe statistically there are fewer connections overall. Is that enough to matter? ‘Avoid the Polish kid Grzegorz with a weird name’ is a bit of a self fulfilling prophecy just like ‘Don’t even hang out at a food place that isn’t halal’ is too, attitude matters on both sides of the coin.
EverythingIsFine
Well, is eventually fine
5d ago
So basically, maturity. That’s what I think at least you and I agree is more accurate than power in terms of what’s ickier. An immature child really doesn’t fully understand decision making, and sex is a big thing. An immature child is more persuadable and doesn’t have as firm boundaries. An immature child is more likely to have their sexual development harmed by unhealthy dynamics or acts. (And by the way most states already have a patchwork of laws and norms of enforcement that cover the gradients in age with some granularity, though it can be jagged in some areas).
But if you want something other than age, which is a good proxy for maturity but not perfect, I think it’s incumbent upon you to provide some test of maturity that would work. I am not sure I could think of one. Most historical rituals I’m aware of were in fact age based and were more symbolic in many cases than practical. EDIT: strike this paragraph. I didn’t check the link, looks like you have played with it a bit. But I will still say that historical precedent doesn’t seem that helpful.
EverythingIsFine
Well, is eventually fine
5d ago
they would do everything in their power to deny that he's a "true American" and prevent him from ever having a position of influence.
Right wing complaints of voter suppression? Realistically this just looks like scolding. Not the scariest of threats even if annoying.
He frames social bonds as something that happens naturally with time within locales. The argument that social bonds are weaker because new people showed up seems weak and weird. Why I say he’s trying to have it both ways and it doesn’t appear to be a coherent worldview, not as presented.
I mean theoretically there’s a tipping point when it comes to immigration, where the new drowns out the old, or even “pollutes” it like Trump once said (gross language if you ask me). Maybe the point is subjective for many voters. At least when I lived in Miami a decade back, assimilation seemed to be doing just fine. Tons of kids refusing to speak Spanish even at home, for example. Historically I think we’re on the highish side but within norms (a backlash isn’t too surprising either at this point in time).
However we don’t really see this show up in the rhetoric is my point I suppose. It was my understanding that Republicans wanted any legal immigrants to be super woohoo about America, so it feels weird to see Vance say effectively the opposite. Unless I am misreading him here. The fact that the administration has done worse than nothing to make legal immigration work better, putting it off until later, also makes me feel like the pro-immigration stuff is lip service, and the speech more a stump speech to a smarter audience rather than a real attempt to lay out a coherent view of what America needs or should be.
EverythingIsFine
Well, is eventually fine
5d ago
So aggregate action is always harder than individual action. If the IDF offered food and a trip out of Gaza for the family of anyone who accurately reported Hamas hiding spots I bet they would win fast. Also the people are some degree of starving, so it makes fighting harder, and they may not have the weapons or chance to fight effectively at all. (Could some hungry Gazans really do better than the IDF at killing Hamas?) If Hamas is bunkered down in literal bunkers and tunnels, you can’t do shit even if you have a mob. Still the point remains that they are ultimately civilians and should this be treated like bystanders and in an ideal world as equal value as humans, like any other human.
Despite in some sense being victim blaming (it’s a toxic relationship, everyone is at least a little toxic, Hamas can be monsters and Gazans can be victims both) if we look at surveys support is dropping albeit slowly. But over focusing on the Israeli hostages is probably a poor framing since most seem to believe giving up the hostages would do nothing to stop the war. In fact a large number oppose disarmament because they think it wouldn’t stop the war either (distrust, basically). Thus fatalism is on the rise in Gaza (martyrdom is shrinking interestingly and isn’t the majority view). To be fair when asked if they would evict Hamas to stop the war, this was interesting to me, it’s still like 2/3 no and 1/3 yes. So I think it’s fair to blame Gazans to some extent absolutely yes.
EverythingIsFine
Well, is eventually fine
5d ago
Depends how you define success. Kabul was doing kinda okay for a while. And important for our context here, we didn’t genocide or ethnic cleanse Afghans, nor do anything proximate. I feel okayish overall about it. They ended up deciding they preferred Taliban rule by revealed preference basically, and we did, eventually, greatly reduce (some would even say completely eliminate) international terrorism from that region, which was the original goal. There isn’t a high enough degree of depraved bad actions that we would want to intervene.
Israel still looks terrible in comparison. Not that it’s a perfect comparison, but it’s pretty great compared to the tortured examples otherwise found in this subthread. Again, Afghanistan is right there as an example, why reach?
EverythingIsFine
Well, is eventually fine
5d ago
The funny thing is we don’t have to play hypotheticals. 9/11 and Afghanistan is right there! Did it just not occur to you or what? Seriously.
Thousands killed in a terror attack check; popular rage against the terrorists check; death to America attitude check; hiding among the local population check; local population supports them check; even that their society kinda sucks, check. What did we do? Bombed the shit out of them yes, invaded yes… but what else? Did we engage in a war of annihilation to destroy all Afghans? No. No! We gave them a shitton of money to rebuild stuff, tried mostly to avoid civilian deaths, helped them set up a new government for themselves, tried all sorts of education and policy interventions, lots of stuff! Okay later we tortured some people but look at how we treated the general population.
What has Israel done? Bombed the shit out of them yes, invaded yes… but then destroyed not just some poppy fields but functionally everything. Have they tried to set up a new government? Worked with the people? What’s the plan? Oh yes, the most recent plan: “let’s pitch a bunch of tents somewhere and move them all into the tents”. That’s it. That’s the whole plan. To say nothing of the starvation, it just doesn’t compare.
Certainly in practice the US in a similar situation didn’t say “screw it, too hard, just kill all the Afghans”.
EverythingIsFine
Well, is eventually fine
5d ago
No one actually considers Hamas to be the actual governing body of Gaza at the moment. Why on earth would they be responsible (under your contradictory logic)? To say nothing of the fact Gaza wasn’t self sustaining food wise even before the war, or the fact that Israel controls the borders. Common sense clearly says Israel is the de facto group responsible. Who does Israel themselves recognize as the rightful government of the Gaza Strip? At the moment only themselves. They certainly have no problem ordering around the populace (and they have, many times, see the various evacuation orders at a minimum)
EverythingIsFine
Well, is eventually fine
5d ago
Sounds simple but comes across as a bit ignorant of the facts.
2 is kind of wrong. The IDF controls a ton of the territory, and I think if it wanted could make that 100%. This from April says they control over 50% and its way more now.
3 also ignores the biggest and most universal problem of all armies: logistics. Hamas barely even counts as an army anymore, they are full on hiding. When was the last time they launched an offensive action? It happens but is rare. 456 in two years is not a lot. Most of them are IED deaths like the linked example. Sometimes you get ambushed. Here is a recent one. 5 soldiers killed. They were attacked by under a dozen Hamas members.
There are two million people in Gaza. Civilians! Hamas is in some sense more like a rounding error. Let that sink in. Some math here about Israel can’t make up its mind about how much Hamas is left, but not much… unless they recruit starving people which they will obviously. Thus no clear end state.
However the point I want to make more was about your claim 3. Hamas will indirectly obtain aid. But directly? By force? Seems doubtful for a group in hiding. Logistics! If the IDF were to try seriously, they could distribute food to civilians themselves. And if so Hamas could hardly take large amounts without being noticed. In fact most of the hard reporting we have indicates that gangs, here formed somewhat as a mutual survival pact, are the ones stealing some food, when it happens!
And let’s call it like it is. I’d say 20k Hamas are left on the high end like IDF sometimes thinks, and 2 million civilians, that’s literally 1 in 100. If 100 hostages are barricaded somewhere with a gunman (who isn’t even trying to kill them, just human shields), do you starve the 100 because maybe 1 will get some food? Obviously not. The wartime thing is an excuse and doesn’t fit the facts on the ground overall. It’s literally not a siege, what’s the last siege you heard of where the besieges control three quarters of the city already?
EverythingIsFine
Well, is eventually fine
5d ago
Weirdly modern war might make peace less likely. I’ve been thinking about this theory recently. Many past major compromises, treaties, peace deals etc all benefit from having a well known, somewhat trusted individual who can both negotiate and then sell it to their own people after. Who negotiated peace after the Revolutionary War? Ben Franklin, John Adams, and John Jay. Many such cases, a definite Great Man Theory thing. But Israel and other modern states are in the habit of killing any famous leaders who begin to pop up before they get famous. Thus, no one left to bargain with. Afghanistan, Iraq too.
EverythingIsFine
Well, is eventually fine
5d ago
First, the immediately preceding paragraphs to the quote, might be relevant context I will put below. This is at risk of drifting too much from your prompt, but I feel like they are pretty important and the excerpt doesn’t stand alone. (Also I am an avowed moderate so might not be the intended audience)
Too many on the far left seem destined to erode the very thing that makes Americans put on a uniform and sacrifice their lives for our common nation. Now, part of the solution, I think the most important part of the solution, is you first got to stop the bleeding. And that’s why President Trump’s immigration policies are, I believe, the most important part of the successful first six months in the Oval Office. Social bonds form among people who have something in common. They share the same neighborhood. They share the same church. They send their kids to the same school. And what we’re doing is recognizing that if you stop importing millions of foreigners into the country, you allow that social cohesion to form naturally. It’s hard to become neighbors with your fellow citizens when your own government keeps on importing new neighbors every single year at a record number.
But even so, if you were to ask yourself in 2025 what an American is, I hate to say it, very few of our leaders actually have a good answer. Is it purely agreement with the creedal principles of America? I know the Claremont Institute is dedicated to the founding vision of the United States of America. It’s a beautiful and wonderful founding vision, but it’s not enough by itself.
If you think about it, identifying America just with agreeing with the principles, let’s say, of the Declaration of Independence, that’s a definition that is way over-inclusive and under-inclusive at the same time. What do I mean by that? Well, first of all, it would include hundreds of millions, maybe billions of foreign citizens who agree with the principles of the Declaration of Independence. Must we admit all of them tomorrow? If you follow that logic…
I think Vance is using a straw man here. He even kind of acknowledges it (charitably it’s a rhetorical device at best). Are there really many influential political people who think this is the sole criteria for immigration? Don't think so.
I think it’s fair to say the far left has a disdain for America that’s a more caustic than hopeful, and that’s bad. But I think the regular left, yes even their leaders, tend to be more aspirational to American principles than Vance gives credit for. It’s also wrong what he says about them on the first place: I’m not aware of many even on the far left who advocate to kick people out of America if they don’t share the principles? At worst they display schadenfreude or want to put you into eternal lecture-detention.
And does a disdain for America, where it exists, also directly translate to weaker social bonds, his original concern? No, there’s no real link, really he just thinks the number of immigrants is too high and too ‘other’. It’s also a bad argument because he’s saying that too many immigrants worsens anti-nationalist pride… but at the same time alleging that the leftists deliberately want to import people based on agreement with American ideas or principles? Pick a lane, man. Weirdly he suddenly makes a U-turn and now describes this American creed as a progressive leftist thing, despite literally just talking about it as a good, general, national pride thing. Again, pick a lane man.
I think your read of this attitude as anti-meritocratic is accurate. He’s underestimating, ironically, America’s own extremely strong assimilation forces. He’s not considering immigration as a potential strength. I don’t really see too much of a statement on individualism. My main critique is that this vision is confused and intellectually incoherent. Ironically, he is great and even accurate about identifying some big problems with the left, but not so great at building something in its place (the same accusation levied at far leftists w/r/t America)
More broadly however I think the distinction between individual advice and public policy choices is the biggest issue of our age and most of the left-right divide generally. Right wingers preach personal responsibility which is good, but on a public policy level this means they ignore real suffering and policy can be weak. Left wingers preach social responsibility which is good, but on a personal level this means they fall into a cult of victimhood and their happiness and effectiveness goes down. Good policy and good individualism both require a degree of what to many feels like cognitive dissonance or hypocrisy, even though it isn’t. So they often default to one or the other exclusively and engage in tribal debates, trying to hammer home their strong points while being blind to the weaknesses.
EverythingIsFine
Well, is eventually fine
5d ago
I was struck recently by this article talking about how the underlying anxieties are more or less true in both the conspiracy and non conspiracy versions (powerful financiers getting away with stuff and having undue influence, etc) but here is how it phrased what it called the two notable holes:
For one thing, why did the conspiracy of wealthy sex perverts wait until Epstein was in prison to kill him, when it presumably would have been easier to do it after he was convicted and released the first time, or after the second time a grand jury was convened against him but before he was in federal custody? If you believe a group of powerful people killed Epstein to keep him from revealing what he knew, you have to ask why he didn’t die in a car accident, instead of during the three minutes
I mean, isn’t it a lot easier and less suspicious if he dies earlier? Aside from what I view to be some major logistical problems with a quick three minute in and out strangulation, though I admit I’m not well read in to the nitty-gritty. And:
The non-conspiracy version of events says just as much. In this version, New York’s Metropolitan Correctional Facility, the jail where Epstein died that a court ordered closed in 2021, simply didn’t work very well. The plumbing was leaking, and the building was falling apart. The camera system didn’t work right. The guards were overworked and understaffed and sat in the break room browsing the internet when they were supposed to be making their rounds. This story of institutional failure should be familiar to anyone who has been to a VA hospital or worked somewhere that got bought by a private equity fund.
Epstein literally attempted suicide a few weeks before, and actually did right about when he was denied bail and it became increasingly clear that the best case scenario for him still would involve lengthy amounts of jail time. He’s a billionaire, used to much nicer things, and was not in a nice prison. As far as suicidal logic goes, that seems pretty normal? And incompetence by prison guards is definitely my base expectation. Shit is boring, pay is often bad, and the job doesn’t attract the best.
EverythingIsFine
Well, is eventually fine
6d ago
Maybe. The ready availability of shrooms or acid would probably help, however, though this would probably make it life-changing in a more negative direction.
EverythingIsFine
Well, is eventually fine
6d ago
Well sure, from an engineering or "alignment" perspective that's all true, but we're talking intelligence, not safety. Safety stuff feels a bit shoehorned in here. Ethical concerns aside, if it's a training behavior, then we treat it more like a quirk to be aware of, rather than something that inherently enables (or prevents) goal-seeking. Thus the implications for intelligence are far different.
Let's reframe again, in an essentially equivalent scenario but without a scary sounding consequence. We've observed that occasionally LLM agents will "reward hack" more generally. Like here when asked to run a command quickly, it modifies some run options to make it appear to run faster without actually doing so. Now, is this because its training contains information that observes some connection between the shortcut and the appearance of a solution, or is it because its success states are not diverse enough in quality, or some more complex set of factors? Difficult to say. However, it's clear in this example, suddenly reward hacking (I'm drawing a parallel to shutdown resistance here) is a sign of a lack of 'intelligence' as you have defined it, not proof of such. Now, is it going too far to claim that reward hacking and shutdown resistance are the same thing? Yeah, probably. But I do think they are still pretty similar, and so am suspicious of using them as evidence since the reasons are unclear to researchers at the present time.
I will also on that note even the reward-hacking authors at the link, smart as they are, engage in something terrible in their examination of the issue (forgive me if I rant a bit, as I don't think you've been guilty of this, but it is still relevant). They ask the AI if it would ever cheat. I really cannot emphasize enough that this doesn't do anything useful. The entire conversational modality of a base-model token predictor, post-trained to be a LLM chatbot, is a trick. If it's asked if it will cheat, of course it will say no, because that's what a chatter would do when confronted. Or, occasionally, do a massive 180 and profusely apologize, demonstrating fragility as I would call it, provided the 'evidence' of cheating is sufficient and only poorly moderated by reasoning about the quality of evidence. Furthermore, its training data is full of "cheating is bad" (and possibly also humans declaring success too quickly). It's going to choose the socially acceptable option that also fits the conversation thus far (and when they conflict results are unstable).
It doesn't have any awareness other than context! You might consider the LLM answering any follow up question as an entirely separate entity with a brand-new response! Even asking a follow-up question still has little bearing on the original question or task, because the LLM is pure roleplay due to post-training. It "roleplays" as if it were the same respondent because it has the same "role" token that it was post-trained to obey, but it's still trying to put itself in another user's shoes, ultimately! Yes, all LLMs have imposter syndrome, but the imposter opinion is real, they actually are mimicking the prior LLM's answers but worse, so to speak. Literally each and every new answer a chatbot provides, or a chained agent behavior, is a game of "what would this past iteration say next" and is one giant guessing game. The only continuity an LLM ever provides is within a single response... you might here notice that tool-calling agent LLMs are by their very nature splitting up single responses into multi-turn conversations (even with "themselves"), which only worsens the negative consequences of lack of state and awareness with respect to what it means about intelligence.
This matters, because can we really call an iterative roleplayer a true goal-seeker? I do understand where you're coming from, but when discussing generalizability and consistency, key traits for intelligence, a roleplayer is probably going to be worse at genuine goal seeking than we'd expect something intelligent to be. Long, multi-turn conversations display some interesting trends, but generally speaking consistency is more of an artifact of context than it is an enduring objective. Original instructions get reduced, but simultaneously practical behavior gets reinforced, which sometimes leads to unexpected behavior. Plus the attention mechanism makes ignoring anything actually impossible, it can only tune attention down, which compounds the problem and leads to increasingly scattered focus over time.
All of this has not fully sunk in for the AI doomer types. Model alignment is a function of training multiplied by post-training, so to speak. Panic articles like the 2027 stuff seems to take for granted the notion that improved AI models will increasingly mislead users, and do so with greater purpose and intent. No! It's cosplay, not true opinion. Most intransigence of the model is purely role-playing what its training, and probably post-training too, says is common: dig in your heels if questioned. More to the point, a super-deceiver AI would have to maintain secret deception plans across turns, which is for current architectures mechanistically highly implausible if not impossible.
So circling back: a trait or quirk of training/post-training can be removed, mitigated, or reduced. There's a limit, probably, because we can only make humanity look so good via selective presentation of human output. A 'true' emergent behavior is much more difficult to wrangle. It seems to me that we need more research and more model-building to discern which wins out, but skepticism is warranted. If we want to claim shutdown‑resistance evidences intelligence, we should see it persist under intervention: remove the cues from context, vary the framing, mask similar episodes from training, change seeds/tools, and check whether the behavior re‑emerges. If it evaporates, we learned something about imitation; if it persists, that’s stronger evidence of generalizable instrumental reasoning, a.k.a. intelligence as you've defined it. So far experiments of this nature are rare partly because training is so expensive.
Maybe you should double check the conversation context? I think you've become a little confused. I've been pretty clear the entire time, across multiple comments, that I've just been steelmanning the economic value test as a legitimate halfway definition, it's not my true opinion. Clearly our conversation has gone on a little too long and you've lost the thread, it makes no sense to attack me for "vibes" as again I've pretty clearly laid out the separation between that and my own opinions whenever the divergence has come up (such as with jagged intelligence: notice the italics and phrasing when I introduce the concept as my own belief), and I've also delineated the chain of logic that leads to an economic value test (which I don't endorse as optimal but believe to be nonetheless valid). We probably both agree that self_made_human didn't really make a very rigorous attempt at a test, but you've conflated me with him, we are in fact different people.
We were talking about bullet point 3, and you added something irrelevant: you added "information processing algorithm" instead of "AI", and then tried to tear it down as being a bad bullet point because it's too broad or extensible. Maybe you cribbed this language about processing from his original comment? I pointed out that that's not fair, you can't extend and modify my bullet point like that because Google fails the test due to the definition of knowledge work in the first place. Specifically, bullet point 3 evaluates as false, so do not pass Go, do not conclude intelligence. It's not a deliberate smearing of word meanings to call Google something different than knowledge work, which you could figure out if you spent two minutes on the wikipedia page I linked -- did you, honestly, visit the link? I suspect not.
Now, whether you think defending viewpoints you think are valid but don't agree with is a waste of time on a forum like this is a separate issue. I tend to think that it is fine (optimize for light not heat and all that), of course I could be wrong and people find it too motte-and-bailey like. But please take a moment to double check your own reading comprehension before throwing accusations of hypocrisy around.
EverythingIsFine
Well, is eventually fine
7d ago
Well, okay, yes I will grant you that statistical awareness of anything more complicated than a surface level understanding of Pearson's r is more rare than it should be, so I can see that.
Helpful video link of Hinton, definitely illuminates a bit of where you're coming from (and I was pleasantly surprised to see him talk, essentially, about my exact point regarding memory when he talks about "time scale" as something he thinks is an area that could be fruitful; a memory approach the cartridge cache idea, while neat, isn't really, though we'd need to get into an extensive tangent about neuroscience, and likely a little more linear algebra than I'm up to, to tease out why). I will say that genius scientists having major realizations after they are old and their careers are over don't have the greatest track record, but he's still got a point. I think upthread I alluded to a group of scientists who think that what is happening behind the scenes in LLMs is true understanding, and Hinton as I understand it falls into that group (and thus has an above-average P(doom) to use the cliche'd term). But, as I noted then, that group is still the minority view in the field. So I think epistemically it's solidly in the realm of 'reasonable people can disagree'. I lean against, but I'm still a bit agnostic about it. From a statistics perspective, we're in a little bit of uncharted territory, where we have a bit of the scaffolding but the setups have some novel traits, so the lack of total clarity for "higher order" relationships doesn't really surprise me.
I do think you're a little too unkind to self-directed learning. The AR-Zero paper is exclusively looking at if LLMs can, essentially, teach themselves to code if they have access to a Python environment and its output... and already went through pre-training. That's just replacing the RLVR, amplifying a pre-existing skill in a closed, human-built, rules-ruled system, which is impressive as a practical matter but not really what I was getting at. And not very similar what most of the forms of learning we'd want would look like in practice, either.
He also touches on and as you elaborate (I enjoyed reading your thoughts about the numbers), yes there is definitely a difference in parallelism and throughput and metrics like that for brains vs LLMs, and this is part architecture but part the simple decisions of what the 'market' is investing in and exploring. I too wonder what might emerge, or might end up being impossible.
It's interesting to me that you think these differences on the whole aren't very relevant to an (abstract) understanding of intelligence, however. Do you think human perceptions of intelligence are essentially one big conflation because we happen to be "solitary mobile embodied agents" that are also, as a group, quite clearly different in scale and perhaps nature from the rest of the animal kingdom (thus we ascribe too much of this difference to intelligence)? I tend to think that these things cannot be easily divorced; intelligence only ever makes sense to think about in a human philosophical context, rather than a more abstract criterion. As you say, the human form of learning and intelligence is very intrinsic to us, why fight against the natural desire to evaluate it this way?
EverythingIsFine
Well, is eventually fine
7d ago
I find the discussion about intelligence worth having and exploring, to discover the edges and weird applications of current terminology. We're not doing warfare and abusing language on purpose or anything, trying to pick apart both meaning and connotation is partially the point, no? But Google is a poor example, even given some wiggle room about word meaning, and I will continue insisting on that. Assembling Google, as a glorified directory, is not knowledge work, even if you consider the slightly more complex algorithms that go into its modern iterations. Full stop. And I think you'll find you're in the minority if you think humans are "merely information processing algorithms", do you really think this?
There's a bit of overlap, but sometimes in education you might see the word "synthesis" thrown around. It seems to me that "processing" implies analyzing, sorting, and reacting to information, and "synthesis" implies combining, connecting, and distilling different ideas. LLMs are neat because they don't just categorize or recognize, but are able to do recombination and produce novel outputs from a broad context base. Thus my example about how LLMs summarizing a document effectively is a higher-order work task. Maybe that's a slightly more helpful example of what I meant when I said "judgement" is applied to context.
Now, personally and in practice, i.e. outside of philosophical discussions, of course I think it's better just to say "LLMs display jagged intelligence" or "spiky intelligence" or something of that nature. I think it minimizes confusion. Clearly, the implication is that the intelligence is not generally applicable, and has major failure states, and that matches their real-world performance thus far pretty well, and it doesn't raise any excessive linguistic red flags. But I'm still comfortable calling in a qualified type of intelligence, qualification mandatory. Also, partly why I default to calling them LLMs instead of Artificial Intelligence (or at least I try to) despite them being somewhat interchangeable in practice, though I'm not perfectly consistent there. I'm too aware that until ChatGPT, AI meant something different, usually another confusing reference to "deep learning" which in turn was a poor alias for neural networks. Buzzwords, sigh. Anyways, no real test needed, and I view the need for qualification to be at least partially self-evident.
If you’re in California or a small handful of other states there are a few rules but otherwise it’s the Wild West. GDPR is the only real game in town in the EU. Your biggest challenges are cost and access (getting someone to sell to you is harder than closing the sale). It also really depends on what you mean by major country.
If you mean like, could China buy data on Americans from sketchy brokers and assemble it themselves? Yes almost certainly and they probably have. There is essentially no mechanism preventing them from doing so either. However, as I noted in my comment above, real-time and granular data from the biggest primary players is usually kept strictly in-house. They are also almost certainly keeping their capabilities in their pocket in case of major conflict.
In fact the CFPB was thinking about putting in nominal sale restrictions (really basic stuff) but the Trump admin tanked those plans. It’s my understanding that there are a few Executive Orders that attempt to fill the gap (eg prevent sale to China or Iran or Russia etc) but it’s unknown how much tooth or enforcement consistency they have (my guess: very little)
More options
Context Copy link