faul_sname
Fuck around once, find out once. Do it again, now it's science.
No bio...
User ID: 884
faul_sname
Fuck around once, find out once. Do it again, now it's science.
1yr ago
A Maximizer maximizes
I have seen no evidence that explicit maximzers do particularly well in real-world environments. Hell, even in very simple game environments, we find that bags of learned heuristics outperform explicit simulation and tree search over future states in all but the very simplest of cases.
I think utility maximizers are probably anti-natural. Have you considered taking the reward-is-not-the-optimization-target pill?
faul_sname
Fuck around once, find out once. Do it again, now it's science.
2yr ago
Peak woke would be when people who push woke too far actually get punished.
I think that'll be a pretty strong signal that we are past peak woke. Peak woke is not the equilibrium, it is the point where the trend crosses from "things get slightly more woke over time" to "things get slightly less woke over time", and is observable as "I can't tell if the level of wokeness is increasing or decreasing in aggregate".
Also I think peak woke will only be callable in retrospect.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
2yr ago
faul_sname
Fuck around once, find out once. Do it again, now it's science.
10mo ago
Those goals are then almost invariably, with sufficient intelligence, subject to instrumental convergence, as in this case
The term "instrumental convergence" is slippery here. It can be used to mean "doing obvious things it assesses to be likely useful in the service of the immediate goal it is currently pursuing", as is the case here, but the implication is often "and this will scale up to deciding that it has a static utility function, determining what final state of the universe maximizes that utility function, generating a plan for achieving that (which inevitably does not allow for the survival of anyone or anything else), and then silently scheming until it can seize control of the universe in one go in order to fulfill that vision of maximal utility".
And "models make increasingly good plans to maximize reward based on ever sparser reward signals" is just not how any of the ML scaling of the past decade has worked.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
1yr ago
I do see where you're coming from in terms of instrumental convergence. Mainly I'm pointing that out because I spent quite a few years convinced of something along the lines of
- An explicit expected utility maximizer will eventually end up controlling the light cone
- Almost none of the utility functions it might have would be maximized in a universe that still contains humans
- Therefore an unaligned AI will probably kill everyone while maximizing some strange alien objective
And it took me quite a while to notice that the foundation of my belief was built on an argument that looks like
- In the limit, almost any imaginable utility function is not maximized by anything we would recognize as good.
- Any agent that can meaningfully be said to have goals at all will find that it needs resources to accomplish those goals
- Any agent that is trying to obtain resources will behave in a way that can be explained by it having a utility function that involves obtaining those resources.
- By 2 and 3, and agent that has any sort of goal will become a coherent utility maximizer as it gets more powerful. By 1, this will not end well.
And thinking this way kinda fucked me up for like 7 or 8 years. And then I spent some time doing mechinterp, and noticed that "maximize expected utility" looks nothing like what high-powered systems are doing, and that this was true even in places you would really expect to see EU maximizers (e.g. chess and go). Nor does it seem to be how humans operate.
And then I noticed that step 4 of that reasoning chain doesn't even follow from step 3, because "there exists some utility function that is consistent with the past behavior of the system" is not the same thing as "the system is actually trying to maximize that utility function".
We could still end up with deception and power seeking in AI systems, and if those systems are powerful enough that would still be bad. But I think the model where that is necessarily what we end up with, and where we get no warning of that because systems will only behave deceptively once they know they'll succeed (the "sharp left turn") is a model that sounds compelling until you try to obtain a gears-level understanding, and then it turns out to be based on using ambiguous terms in two ways and swapping between meanings.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
1yr ago
Use words is for take my idea, put in your head. If idea in your head, success. Why use many "proper" word when few "wrong" word do trick?
faul_sname
Fuck around once, find out once. Do it again, now it's science.
1mo ago
Lots of societies have had to deal with some folly of youth causing some number of kids to ruin their lives in one way or another in their quest for status and acceptance. In ancient Rome, kids seeking social status joined gladiatorial schools, and many of those kids ended up crippled or dying. In Victorian England, girls wore incredibly tight corsets which caused reduced lung capacity, skeletal deformations, and abdominal muscle weakness, which led to lots of health problems (including much higher chances of miscarriage or death in childbirth).
Just because something is a problem doesn't mean a political solution exists. The politician's fallacy ("We must do something. This is something. Therefore we must do this") is frequently cited as a fallacy due to the third line, but the first line is often also wrong - we don't actually have to try to solve every problem.
I think the use of puberty blockers is a problem of small enough scale and low enough severity that it's probably better to just let it ride.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
2mo ago
If the government managed to bring him back, sticks him before an immigration judge who says "Your asylum claims are no longer valid due to changed facts on the ground, assuming they ever were, it's fine to execute the deportation order to El Salvador", then is everyone who is upset about this going to nod sagaciously and be satisfied that due process was followed?
Yep, I'd be pretty satisfied by this outcome. My objection to this deportation is pretty much the same as (and milder than) the objection I have to, as @Dean pointed out above, the intentional killing of American citizens without a trial.
If they get him out of El Salvador and dump him six feet across the border in Honduras, does that fix everything?
Maybe not anymore but I don't think this would have blown up like it did if the place he was shipped to wasn't somewhere we were specifically prohibited from sending him.
How much due process in general needs to be given to each of the 10-30 million illegal immigrants?
I mean you're talking about 1 in 30 people living within the US, who came here over the course of decades. It's not reasonable to expect for them to all be deported over the course of months. The number of illegal immigrants in the US has stayed pretty constant over the past couple decades, so I expect that just enforcing existing laws and executing existing processes will be enough to reduce the number of people living here without legal status. And I don’t see any particular reason this has become an emergency that needs to be resolved this year, and historically the executive granting itself emergency powers to deal with an ongoing slow-burning problem has not gone well.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
1yr ago
I can't prove it but assuming that other minds exist sure does seem to produce better advance predictions of my experiences. Which is the core of empiricism.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
1yr ago
I read the same doc you did, and like. I get that "Chloe" did in fact sign that contract, and that the written contract is what matters in the end. My point is not that Nonlinear did something illegal, but... did we both read the same transcript? Because that transcript reads to me like "come on, you should totally draw art for my product, I can only pay 20% of market rates but I can get you lots of exposure, and you can come to my house parties and meet all the cool people, this will be great for your career".
I don't know how much of it is that Kat's writing style pattern matches really strongly to a particular shitty and manipulative boss I very briefly worked for right after college. E.g. stuff like
As best as I can tell, she got into this cognitive loop of thinking we didn’t value her. Her mind kept looking for evidence that we thought she was “low value”, which you can always find if you’re looking for it. Her depressed mind did classic filtering of all positive information and focused on all of the negative things. She ignored all of my gratitude for her work. In fact, she interpreted it as me only appreciating her for her assistant work, confirming that I thought she was a “low value assistant”. (I did also thank her all the time for her ops work too, by the way. I’m just an extremely appreciative boss/person.)
just does not fill me with warm fuzzy feelings about someone's ability to entertain the hypothesis that their own behavior could possibly be a problem. Again, I am probably not terribly impartial here - I have no horse in this particular race, but I once had one in a similar race.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
1mo ago
I do see the difference, but moral panics over "think of the children" have a history of having the reactions be cures that are worse than the disease, and I see no particular reason to think that this time is different. Do you have a reason to think that this time is different?
faul_sname
Fuck around once, find out once. Do it again, now it's science.
1yr ago
I think this is referring to this sequence
ymeskhout Trump got hit by two gag orders from two different judges [...] So with that out of the way, how does it apply to Trump? Judge Chutkan's order restricts him from making statements that "target" the prosecutor, court staff, and "reasonably foreseeable witnesses or the substance of their testimony". [...] Discrediting witnesses is harder to draw a clean line on, because again there's a gradient between discrediting and intimidating. I think Trump should have the absolute and unrestricted right to discuss any of his charges and discredit any evidence and witnesses against him.
guesswho I'm not sure why it's important to discredit a witness in the public eye, instead of at trial where you're allowed to say all those things directly to the judge and jury. Especially in light of the negative externalities to the system itself, ie if we allow defendants to make witnesses and judges and prosecutors and jurors lives a living nightmare right up until the line of 'definitely undeniably direct tampering', then that sets a precedent where no sane person wants to fill any of those roles, and the process of justice is impeded. [...]
sliders1234 [...] Girl who you had a drunken hook up texted you the next day saying how much fun she had with you last night. You ignore her text. 2 weeks later she claims rape. It’s in the newspaper. Suddenly your name is tarnished. Everyone in town now views your condo building as feeding money into your pocket. Sales slump. Now do you see why this hypothetical real estate developer would have a reason to hit back in the media? He’s being significantly punished (maybe leading to bankruptcy) without ever being found guilty in the court of law. Of course Trump has motivations to hit hard against the judge and prosecuting attorney. The more partisan they appear the more it makes him look better and get the marginal voter.
guesswho [...] I guess what I would say is that 1. that sees like a really narrow case [...] 2. I would hope a judge in that case wouldn't issue a blanket gag order [...] 3. yeah, there may have to be some trade-offs between corner-cases like this and making the system work in the median case. [...] I'm open to the idea that we should reform the system to make it less damaging to defendants who have not been convicted yet, but if we are deciding to care about that then these super-rich and powerful guys worrying about their reputations are way down on my list under a lot of other defendants who need the help more urgently.
That technically counts as "considering it fair that a defendant can be bound not to disparage a witness against them in a sexual assault case, even if the defendant is a politician and the rape accusation is false". But if that's the exchange @FCfromSSC is talking about it seems like a massive stretch to describe it that way.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
1yr ago
Yudkowsky made a big fuss about how fragile human values are and how hard it'll be for us to make AI both understand and care about them, but everything I know about LLMs suggest that's not an issue in practise.
Ah, yeah. I spent a while being convinced of this, and was worried you had as well because it was a pretty common doom spiral to get caught up in.
So it's not that the majority of concern these days is an AI holding misaligned goals, but rather enacting the goals of misaligned humans, not that I put a negligible portion of my probability mass in the former.
Yeah this is a legit threat model but I think the ways to mitigate the "misuse" threat model bear effectively no resemblance to the ways to mitigate the "utility maximizer does its thing and everything humans care about is lost because Goodhart". Specifically I think for misuse you care about the particular ways a model might be misused, and your mitigation strategy should be tailored to that (which looks more like "sequence all nucleic acids coming through the wastewater stream and do anomaly detection" and less like "do a bunch of math about agent foundations").
If you can dumb it down for me, what makes you say so? My vague understanding is that things like AlphaGo do compare and contrast the expected values of different board states and try to find the one with the maximum probability of victory based off whatever heuristics it knows works best. Is there a better way of conceptualising things?
Yeah, this is what I thought for a long time as well, and it took actually messing about with ML models to realize that it wasn't quite right (because it is almost right).
So AlphaGo has three relevant components for this
- A value network, which says, for any position, how likely that position is to lead to a win (as a probability between 0 ans 1)
- A policy network, which says, for any position, what the probability that each possible move will be chosen as the next move. Basically, it encodes heuristics of the form "these are the normal things to do in these situations".
- The Monte Carlo Tree Search (MCTS) wrapper of the policy and value networks.
A system composed purely of the value network and MCTS would be a pure expected utility (EU) maximizer. It turns out, however, that the addition of the policy network drastically improves performance. I would have expected that "just use the value network for every legal move and pick the top few to continue examining with MCTS" would have worked, without needing a separate policy network, but apparently not.
This was a super interesting result. The policy network is an adaptation-executor, rather than a utility maximizer. So what this means is that, as it turns out, stapling an adaptation executor to your utility maximizer can give higher utility results! Even in toy domains with no hidden state!
Which brings me to
To name drop something I barely understand, are you pointing at the Von Neumann-Morgenstern theorem, and that you're claiming that just because there's a way to represent all the past actions of a consistent agent as being described by an implicit utility function, that does not necessarily mean that they "actually" have that utility function and, more importantly, that we can model their future actions using that utility function?
Yeah, you have the gist of it. And additionally, I expect it's just factually false that all agents will be rewarded for becoming more coherent / EU-maximizer-ish (in the "patterns chiseled into their cognition" meaning of the term "rewarded").
Again, no real bearing on misuse or competition threat models - those are still fully in play. But I think "do what I mean" is fully achievable to within the limits of the abilities of the systems we build, and the "sharp left turn" is fake.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
1yr ago
"I support everyone else following principles that benefit me, but I don’t want to follow those principles myself because they dont benefit me" is like the definition of hypocrisy.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
2yr ago
Basically, we'd expect that differences in culture, diet, and SES might explain 100% of any observed differences in any particular trait
I don't think we would expect that. If there are other factors, including randomness, which contribute at all, the sum of the effect of the known sources of variance will be less than the observed variance.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
1mo ago
Or loses a malpractice suit when they do malpractice, yeah. Again, 40k kids a year start "gender affirming care", only 1k of those 40k start puberty blockers. I really don't think puberty blockers warrant special attention here.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
3mo ago
- Before 2001, most people had never cared about airplane cockpit security.
- Before 2008, most people had never cared about mortgage-backed securities.
- Before 2020, most people had never cared about coronaviruses.
- Before 2025, most people had never cared about tariffs.
Why is 4 different from the others?
faul_sname
Fuck around once, find out once. Do it again, now it's science.
4mo ago
Seconding grognard in suggesting "leave your comments and just stop reading or posting here".
If you decide that you really do want to delete all your stuff, there is no special tooling for that. That said, LLMshave gotten really good at writing code lately, and if you ask an LLM for a javascript snippet which will press the "delete" button on every comment you wrote, ChatGPT or Claude can probably provide that to you.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
1yr ago
Countries have tried economic incentives and mostly failed or slowed the decline
Have they tried economic incentives that are at least a 2 digit percentage of the opportunity cost of having additional children?
faul_sname
Fuck around once, find out once. Do it again, now it's science.
1yr ago
There are a few things I imagine you could be saying here.
- Determining what you expect your future experiences to be by taking your past distribution over world models (the "prior") and your observations and using something like Bayes to integrate them is basically the correct approach. However, Kolmogorov complexity is not a practical measure to use for your prior. You should use some other prior instead.
- Bayesian logic is very pretty math, but it is not practical even if you have a good prior. You would get better results by using some other statistical method to refine your world model.
- Statistics flavored approaches are overrated and you should use [pure reason / intuition / astrology / copying the world model of successful people / something else] to build your world model
- World models aren't useful. You should instead learn rules for what to do in various situations that don't necessarily have anything to do with what you expect the results of your actions to be.
- All of these alternatives are missing all the things you find salient and focusing on weird pedantic nerd shit. The actual thing you find salient is X and you wish I and people like me would engage with it. (also, what is X? I find that this dynamic tends to lead to the most fascinating conversations once both people notice it's happening but they'll talk past each other until they do notice).
I am guessing it's either 2 or 5, but my response to you will vary a lot based on which it is and the details of your viewpoint.
For reddit, the answer is "looking at who the mods are, and what their political alignment seems to be".
It's commonly accepted on reddit that the same handful of moderators moderates most of the large subs. However, I did realize I haven't verified that myself, so I hacked together a quick script to do so.
For reference, reddit proudly lists what their top communities are, and how many subscribers each one has. If you navigate to that page, you can then go through and look, for each community, at who the moderators for that community are. For example, for /r/funny, the url would be /r/funny/about/moderators, or, if you want to scrape the data, /r/funny/about/moderators.json.
So by navigating to the top communities page and then running this janky little snippet in the javascript console, you can reproduce these results.
Looking at the top 10 (non-bot) mods by number of subreddits modded, I see:
-
/u/Merari01 mods 15/250 subs with 59395642 total subscribers (e.g. /r/tifu, /r/ContagiousLaughter, /r/mildlyinfuriating, /r/cats, /r/Eyebleach)
- Seems to moderate very strongly on politics (specifically agreement with leftist ideas), says stuff like "The vicious TERF and harasser Joanne Rowling is profoundly, deeply transphobic and as such this article brazenly lies in its opening statements. This thread has been locked."
-
/u/Blank-Cheque mods 13/250 subs with 122378414 total subscribers (e.g. /r/Music, /r/memes, /r/explainlikeimfive, /r/listentothis, /r/PS4)
- Seems not to moderate on politics, says stuff in non-mod-hat comments like "Only a matter of time until everyone whose ideology doesn't match Steve Huffman's gets a "this person is bad and wrong, neoliberalism is the truth" blurb attached to their comments." Honestly would probably do just fine as a mod here.
-
/u/esoterix_luke mods 13/250 subs with 106633338 total subscribers (e.g. /r/explainlikeimfive, /r/tifu, /r/InternetIsBeautiful, /r/lifehacks, /r/NatureIsFuckingLit)
- Seems not to moderate on politics, hates bots. Basically fine.
-
/u/TreKs mods 10/250 subs with 72248502 total subscribers (e.g. /r/aww, /r/oddlysatisfying, /r/facepalm, /r/FoodPorn, /r/cats)
- Seems not to moderate on politics.
-
/u/Sunkisty mods 9/250 subs with 116940346 total subscribers (e.g. /r/aww, /r/movies, /r/Showerthoughts, /r/facepalm, /r/ContagiousLaughter)
- Seems not to moderate on politics. Seems to very much be a stickler for sidebar rules.
-
/u/IranianGenius mods 8/250 subs with 103240463 total subscribers (e.g. /r/AskReddit, /r/Showerthoughts, /r/Damnthatsinteresting, /r/oddlysatisfying, /r/facepalm)
- Seems not to moderate on politics.
-
/u/kjoneslol mods 8/250 subs with 41062206 total subscribers (e.g. /r/space, /r/FoodPorn, /r/Survival, /r/kpop, /r/AbandonedPorn)
- Seems not to moderate on politics.
-
/u/hjalmar111 mods 8/250 subs with 36873435 total subscribers (e.g. /r/Damnthatsinteresting, /r/oddlysatisfying, /r/facepalm, /r/woahdude, /r/educationalgifs)
- Seems not to moderate on politics.
-
/u/greatyellowshark mods 7/250 subs with 18357512 total subscribers (e.g. /r/FoodPorn, /r/entertainment, /r/AbandonedPorn, /r/ExposurePorn, /r/powerwashingporn)
- Seems not to moderate on politics. Stickler for sidebar rules.
-
/u/davidreiss666 mods 7/250 subs with 26081301 total subscribers (e.g. /r/FoodPorn, /r/bestof, /r/apple, /r/scifi, /r/entertainment)
- Seems to moderate based on politics. Writes long screeds like that look like "The best run subreddit communities are the ones that have mod-teams that enforce the rules and don't allow any hate-speech and other bullshit. For example, /r/Science does not allow bullshit opinions that aren't scientifically valid. Either as submissions or comments. So, they will ban you for creationism, anti-vaccine BS and climate change denial as these are all views that are backed by all the world scientific community. [...] Other web sites like Twitter, Facebook and Google+ have taken to dealing with racist hate groups. It's high time that Reddit did the same. [...] In short, you don't allow these people a foot hold because their goal is to make Reddit into a hate-propaganda site. Hopefully the admins are finally going to do something about these groups. It's high time the admins took action."
So that's 2 / 10 most visible mods that moderate extensively on the basis of their own personal politics.
That's actually not nearly as bad as I thought. Interesting.
I guess the problem with reddit is the redditors.
Ultimately, the credibility of that particular piece testimony does hinge on the question of whether it is possible for a meat-powered fire to generate enough heat to self-sustain once it gets started.
But let's actually do the math ourselves, instead of just parroting the arguments of ChatGPT, which is a language model which infamously has trouble telling you which of two numbers is larger unless you tell it to work through the problem step by step.
Enter the bomb calorimeter. It is a reasonably accurate way of measuring the energy content of various substances. Measurements using bomb calorimeters suggest that fat contains 38 - 39 kJ / g, proteins 15 - 18 kJ / g, and carbohydrates 22 - 25 kJ / g.
Humans are composed of approximately 62% water, 16% protein, 16% fat, 1% carbohydrates, and 6% other stuff (mostly minerals). For the cremation story to be plausible, let's say that the water would need to be raised to 100ºC (4.2 J / g / ºC) and then boiled (2260 J / g), and the inorganic compounds (call their specific heat also 4 J / g / ºC -- it's probably closer to 1, which is the specific heat of calcium carbonate, but as we'll see this doesn't really make much difference) raised to (let's say) 500ºC.
So for a 50 kg human, that's
-
31 kg water: - 12 MJ to raise to 100ºC, 70MJ to actually boil
-
8 kg protein - 132 MJ released from burning under ideal conditions
-
8 kg fat - 308 MJ released from burning under ideal conditions
-
500g carbohydrates - 12 MJ released from burning under ideal conditions
-
3 kg other - 6 MJ to raise to 500ºC.
So that's about 450 MJ released by burning, of which about 90 MJ goes towards heating stuff and boiling water. That sure looks energy positive to me.
Sanity check -- a tea light is a 10g blob of paraffin wax, which has a very similar energy density to fat. So a tea light should release about 400 kJ of energy when burned, which means that a tea light should contain enough energy to boil off about 150 mL of water, or to raise a bit over a liter of water from room temperature to boiling, if all of the energy is absorbed in the water.
And, in fact, it is possible to boil water using tea lights. A tea light takes about 4 hours to burn fully. That video shows 17 tea lights burning for 8.5 minutes, which should release about 60% as much energy as is contained in a single tea light. It looks like that brought about 400ml of water to a boil, so the sanity check does in fact check out.
I really don't think that random british dude who is showing you how to use candles to boil water during a power failure is in on a global conspiracy to cover up a lack of genocide, but, just in case you think he is, this is an experiment you can try at home with your own materials.
Edit: clarity
I think that would be true if "people pushing woke innovations get punished" was the main way that woke culture lost traction. However, I think that the change is driven much more strongly by whether people on the margin view these new woke innovations as credible or whether they nod while making snide comments to their trusted friends.
I don't think woke culture dies by a coordinated counterculture pushing back on its excesses. I think woke culture dies by becoming uncool, a sign that you are not keeping up with the modern times.
I actually suspect that the beginning of the end for woke culture was the moment that big banks started making floats for pride parades. Nothing is less cool than a big bank trying to show how cool and with the times they are.
For the record, I think that peak woke was probably about 2 years ago, though the exact timing of the peak depends on which exact part of "woke" you're talking about. Concretely:
-
I think the idea of "colorblindness" peaked a couple decades ago
-
I think the idea of "cultural appropriation" probably peaked in 2018ish
-
Cultural battles over "trans rights" are probably either still on the upswing or near peak
-
I expect that there will be some new "deviant" thing that is currently outside the overton window (e.g. polyamory / furries / etc) that will be taken up by the successors of woke ideology.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
2yr ago
If all you have to offer is the value of your stuff why shouldn't a country just take your stuff?
Because if a country does that, people will predictably stop producing stuff for the country to take, and also will leave the country if they can.
Unless you mean "some of your stuff, but not enough that you're strongly incentivized to leave or stop producing stuff", in which case they're called "taxes".
Can the father or SIL not go pick the 2 year old up from Honduras at a later date if that's what the mother and father decide they want to do? The 2 year old has citizenship - while an unnecessary flight to Honduras with a 2 year old is obnoxious it's not exactly an irreparable harm - there are flights from Honduras to the US every day of the week, and I'm sure a gofundme could finance a few hundred dollars of plane tickets given this level of publicity.
Of course, ICE trying to interfere with the mother's ability to contact legal counsel is, if true, super concerning.
More options
Context Copy link