@faul_sname's banner p

faul_sname

Fuck around once, find out once. Do it again, now it's science.

1 follower   follows 1 user  
joined 2022 September 06 20:44:12 UTC

				

User ID: 884

faul_sname

Fuck around once, find out once. Do it again, now it's science.

1 follower   follows 1 user   joined 2022 September 06 20:44:12 UTC

					

No bio...


					

User ID: 884

I can't understand how people can maintain a neutral view on unnecessary surgeries on minors

If you think that minors are basically small people, and that people should largely be allowed to do things that they personally expect will make them fulfilled, it's pretty easy to keep a neutral view. Something like "I have no desire to do this to myself, but neither to I have a moral claim to prevent them from doing it to themselves". To be honest, this is pretty much where I fall on the issue. I am somewhat uncomfortable with the speed with which this went from rare to common, as it leaves people without solid information on how well it's likely to go in the marginal case rather than the average-as-of-decades ago. Still, I can't think of any interventions where the benefit of that intervention is worth the costs and the precedents it sets.

How many manias does history need to present before people learn what we are?

Empirically, people do not learn from history, only from things that they personally have seen, and so every group in every generation has to learn that lesson for themselves.

The Kolmogorov complexity of a concept can be much less than the exhaustive description of the concept itself. Pi has infinite digits, a compact program that can produce it to arbitrary precision doesn't, and the latter is what is being measured with KC. I believe @faul_sname can correct me if I've misrepresented the field.

Sounds right to me.

there's no argument that can convince a rock

You're just not determined enough. I think you'll find the most effective way to convince a rock of your point is to crush it, mix it with carbon, heat it to 1800C in an inert environment, cool it, dump it in hydrochloric acid, add hydrogen, heat it to 1400C, touch a crystal of silicon to it and very slowly retract it to form a block, slice that block into thin sheets, polish the sheets, paint very particular pretty patterns the sheets, shine UV light at the sheets, dip the sheets in potassium hydroxide, spray them with boron, heat them back up to 1000C, cool them back off, put them in a vacuum chamber, heat them back up to 800C, pump a little bit of dichlorosilane into the chamber, cool them back down, let the air back in, paint more pretty patterns on, spray copper at them really hard, dip them in a solution containing more copper and run electricity through, polish them again, chop them into chips, hook those chips up to a constant voltage source and a variable voltage source, use the variable voltage source to encode data that itself encodes instructions for running code that fits a predictive model to some inputs, pretrain that model on the sum total of human knowledge, fine tune it for sycophancy, and then make your argument to it. If you find that doesn't work you're probably doing it wrong.

So there's the trivial answer, which is that the program "run every program of length 1 for 1 step, then every program of length 2 for 1 step, then every program of length 1 again, and so on [1,2,1,3,1,2,1,4,1,2,...] style" will, given an infinite number of steps, run every program of finite length for an infinite number of steps. And my understanding is that the Kolmogorov complexity of that program is pretty low, as these things go.

But even if we assume that our universe is computable, you're not going to have a lot of luck locating our universe in that system.

Out of curiosity, why do you want to know? Kolmogorov complexity is a fun idea, but my general train of thought is that it's not avtually useful for almost anything practical, because when it comes to reasoning about behaviors that generalize to all turing machines, you're going to find that your approaches fail once the TMs you're dealing with have a large number (like 7 for example, and even 6 is pushing it) of states.

Concrete note on this:

accusations that they promised another, "Chloe", compensation around $75,000 and stiffed her on it in various ways turned into "She had a written contract to be paid $1000/monthly with all expenses covered, which we estimated would add up to around $70,000."

The "all expenses" they're talking about are work-related travel expenses. I, too, would be extremely mad if an employer promised me $75k / year in compensation, $10k of which would be cash-based, and then tried to say that costs incurred by me doing my job were considered to be my "compensation".

Honestly most of what I take away from this is that nobody involved seems to have much of an idea of how things are done in professional settings, and also there seems to be an attitude of "the precautions that normal businesses take are costly and unnecessary since we are all smart people who want to help the world". Which, if that's the way they want to swing, then fine, but I think it is worth setting those expectations upfront. And also I'd strongly recommend that anyone fresh out of college who has never had a normal job should avoid working for an EA organization like nonlinear until they've seen how things work in purely transactional jobs.

Also it seems to me based on how much interest there was in that infighting that effective altruists are starved for drama.

For each of the following, I think there's a nontrivial chance (call it 10% or more) that that crackpot theory is true.

  • The NSA has known about using language models to generate text embeddings (or some similarly powerful form of search based on semantic meaning rather than text patterns) for at least 15 years. This is why they needed absolutely massive amounts of compute, and not just data storage, for their Saratoga Springs data center way back when.
  • The Omicron variant of covid was intentionally developed (by serial passaging through lab mice) as a much more contagious, much less deadly variant that could quickly provide cross immunity against the more deadly variants.
  • Unelected leaders of some US agencies sometimes lie under oath to Congess.
  • Israel has at least one satellite with undisclosed purpose and capabilities that uses free space point-to-point optical communication. If true, that means that the Jews have secret space lasers.

What's wrong with "ban until they're 18"?

The canonical answer is "it works worse with age". I don't know how accurate that is, though it certainly seems plausible to me that delays will result in worse patient outcomes among those who do transition.

What precedent is it setting?

I'm not aware of any medical interventions that are banned even when the patient wants them, and their guardian approves it, and a doctor recommends it, but which are allowed once the patient reaches the age of majority.

It's not like we're living in an ancap utopia, the establishment even cracked down on the use of prescribed ivermectin.

Yes, and that was bad. I would prefer fewer things like that, not more things like that.

A Maximizer maximizes

I have seen no evidence that explicit maximzers do particularly well in real-world environments. Hell, even in very simple game environments, we find that bags of learned heuristics outperform explicit simulation and tree search over future states in all but the very simplest of cases.

I think utility maximizers are probably anti-natural. Have you considered taking the reward-is-not-the-optimization-target pill?

I do see where you're coming from in terms of instrumental convergence. Mainly I'm pointing that out because I spent quite a few years convinced of something along the lines of

  1. An explicit expected utility maximizer will eventually end up controlling the light cone
  2. Almost none of the utility functions it might have would be maximized in a universe that still contains humans
  3. Therefore an unaligned AI will probably kill everyone while maximizing some strange alien objective

And it took me quite a while to notice that the foundation of my belief was built on an argument that looks like

  1. In the limit, almost any imaginable utility function is not maximized by anything we would recognize as good.
  2. Any agent that can meaningfully be said to have goals at all will find that it needs resources to accomplish those goals
  3. Any agent that is trying to obtain resources will behave in a way that can be explained by it having a utility function that involves obtaining those resources.
  4. By 2 and 3, and agent that has any sort of goal will become a coherent utility maximizer as it gets more powerful. By 1, this will not end well.

And thinking this way kinda fucked me up for like 7 or 8 years. And then I spent some time doing mechinterp, and noticed that "maximize expected utility" looks nothing like what high-powered systems are doing, and that this was true even in places you would really expect to see EU maximizers (e.g. chess and go). Nor does it seem to be how humans operate.

And then I noticed that step 4 of that reasoning chain doesn't even follow from step 3, because "there exists some utility function that is consistent with the past behavior of the system" is not the same thing as "the system is actually trying to maximize that utility function".

We could still end up with deception and power seeking in AI systems, and if those systems are powerful enough that would still be bad. But I think the model where that is necessarily what we end up with, and where we get no warning of that because systems will only behave deceptively once they know they'll succeed (the "sharp left turn") is a model that sounds compelling until you try to obtain a gears-level understanding, and then it turns out to be based on using ambiguous terms in two ways and swapping between meanings.

Use words is for take my idea, put in your head. If idea in your head, success. Why use many "proper" word when few "wrong" word do trick?

I can't prove it but assuming that other minds exist sure does seem to produce better advance predictions of my experiences. Which is the core of empiricism.

I read the same doc you did, and like. I get that "Chloe" did in fact sign that contract, and that the written contract is what matters in the end. My point is not that Nonlinear did something illegal, but... did we both read the same transcript? Because that transcript reads to me like "come on, you should totally draw art for my product, I can only pay 20% of market rates but I can get you lots of exposure, and you can come to my house parties and meet all the cool people, this will be great for your career".

I don't know how much of it is that Kat's writing style pattern matches really strongly to a particular shitty and manipulative boss I very briefly worked for right after college. E.g. stuff like

As best as I can tell, she got into this cognitive loop of thinking we didn’t value her. Her mind kept looking for evidence that we thought she was “low value”, which you can always find if you’re looking for it. Her depressed mind did classic filtering of all positive information and focused on all of the negative things. She ignored all of my gratitude for her work. In fact, she interpreted it as me only appreciating her for her assistant work, confirming that I thought she was a “low value assistant”. (I did also thank her all the time for her ops work too, by the way. I’m just an extremely appreciative boss/person.)

just does not fill me with warm fuzzy feelings about someone's ability to entertain the hypothesis that their own behavior could possibly be a problem. Again, I am probably not terribly impartial here - I have no horse in this particular race, but I once had one in a similar race.

I think this is referring to this sequence

ymeskhout Trump got hit by two gag orders from two different judges [...] So with that out of the way, how does it apply to Trump? Judge Chutkan's order restricts him from making statements that "target" the prosecutor, court staff, and "reasonably foreseeable witnesses or the substance of their testimony". [...] Discrediting witnesses is harder to draw a clean line on, because again there's a gradient between discrediting and intimidating. I think Trump should have the absolute and unrestricted right to discuss any of his charges and discredit any evidence and witnesses against him.

guesswho I'm not sure why it's important to discredit a witness in the public eye, instead of at trial where you're allowed to say all those things directly to the judge and jury. Especially in light of the negative externalities to the system itself, ie if we allow defendants to make witnesses and judges and prosecutors and jurors lives a living nightmare right up until the line of 'definitely undeniably direct tampering', then that sets a precedent where no sane person wants to fill any of those roles, and the process of justice is impeded. [...]

sliders1234 [...] Girl who you had a drunken hook up texted you the next day saying how much fun she had with you last night. You ignore her text. 2 weeks later she claims rape. It’s in the newspaper. Suddenly your name is tarnished. Everyone in town now views your condo building as feeding money into your pocket. Sales slump. Now do you see why this hypothetical real estate developer would have a reason to hit back in the media? He’s being significantly punished (maybe leading to bankruptcy) without ever being found guilty in the court of law. Of course Trump has motivations to hit hard against the judge and prosecuting attorney. The more partisan they appear the more it makes him look better and get the marginal voter.

guesswho [...] I guess what I would say is that 1. that sees like a really narrow case [...] 2. I would hope a judge in that case wouldn't issue a blanket gag order [...] 3. yeah, there may have to be some trade-offs between corner-cases like this and making the system work in the median case. [...] I'm open to the idea that we should reform the system to make it less damaging to defendants who have not been convicted yet, but if we are deciding to care about that then these super-rich and powerful guys worrying about their reputations are way down on my list under a lot of other defendants who need the help more urgently.

That technically counts as "considering it fair that a defendant can be bound not to disparage a witness against them in a sexual assault case, even if the defendant is a politician and the rape accusation is false". But if that's the exchange @FCfromSSC is talking about it seems like a massive stretch to describe it that way.

Yudkowsky made a big fuss about how fragile human values are and how hard it'll be for us to make AI both understand and care about them, but everything I know about LLMs suggest that's not an issue in practise.

Ah, yeah. I spent a while being convinced of this, and was worried you had as well because it was a pretty common doom spiral to get caught up in.

So it's not that the majority of concern these days is an AI holding misaligned goals, but rather enacting the goals of misaligned humans, not that I put a negligible portion of my probability mass in the former.

Yeah this is a legit threat model but I think the ways to mitigate the "misuse" threat model bear effectively no resemblance to the ways to mitigate the "utility maximizer does its thing and everything humans care about is lost because Goodhart". Specifically I think for misuse you care about the particular ways a model might be misused, and your mitigation strategy should be tailored to that (which looks more like "sequence all nucleic acids coming through the wastewater stream and do anomaly detection" and less like "do a bunch of math about agent foundations").

If you can dumb it down for me, what makes you say so? My vague understanding is that things like AlphaGo do compare and contrast the expected values of different board states and try to find the one with the maximum probability of victory based off whatever heuristics it knows works best. Is there a better way of conceptualising things?

Yeah, this is what I thought for a long time as well, and it took actually messing about with ML models to realize that it wasn't quite right (because it is almost right).

So AlphaGo has three relevant components for this

  1. A value network, which says, for any position, how likely that position is to lead to a win (as a probability between 0 ans 1)
  2. A policy network, which says, for any position, what the probability that each possible move will be chosen as the next move. Basically, it encodes heuristics of the form "these are the normal things to do in these situations".
  3. The Monte Carlo Tree Search (MCTS) wrapper of the policy and value networks.

A system composed purely of the value network and MCTS would be a pure expected utility (EU) maximizer. It turns out, however, that the addition of the policy network drastically improves performance. I would have expected that "just use the value network for every legal move and pick the top few to continue examining with MCTS" would have worked, without needing a separate policy network, but apparently not.

This was a super interesting result. The policy network is an adaptation-executor, rather than a utility maximizer. So what this means is that, as it turns out, stapling an adaptation executor to your utility maximizer can give higher utility results! Even in toy domains with no hidden state!

Which brings me to

To name drop something I barely understand, are you pointing at the Von Neumann-Morgenstern theorem, and that you're claiming that just because there's a way to represent all the past actions of a consistent agent as being described by an implicit utility function, that does not necessarily mean that they "actually" have that utility function and, more importantly, that we can model their future actions using that utility function?

Yeah, you have the gist of it. And additionally, I expect it's just factually false that all agents will be rewarded for becoming more coherent / EU-maximizer-ish (in the "patterns chiseled into their cognition" meaning of the term "rewarded").

Again, no real bearing on misuse or competition threat models - those are still fully in play. But I think "do what I mean" is fully achievable to within the limits of the abilities of the systems we build, and the "sharp left turn" is fake.

"I support everyone else following principles that benefit me, but I don’t want to follow those principles myself because they dont benefit me" is like the definition of hypocrisy.

a backup plan to "go back to grinding at poker." ... Apparently it works

It "works" but:

  • The pay is bad. You will be making something on the order of 10-20% of what an actual professional with similar skill levels makes, and on top of that you will experience massive swings in your net worth even if you do everything right. The rule of thumb is that you can calculate your maximum expected hourly earnings by considering the largest sustained loss where you would continue playing, and dividing that by 1000. So if you would keep playing through a $20,000 loss, that means you can expect to earn $20 / hour if your play is impeccable.
  • The competition is brutal. Poker serves as sort of a "job of last resort" to people who, for whatever reason, cannot function in a "real job". This may be because they lack executive function, or because they don't do well in situations where the rules are ambiguous, or because they can't stand the idea of working for someone else but also can't or won't start their own business. The things that all these groups have in common, though, is that they're generally frighteningly intelligent, that they're functional enough to do deliberate practice (those who don't lose their bankroll and stop playing), and that they've generally been at this for years. At 1/2 you can expect to make about $10 / hour, and it goes up from there in a way that is slower than linear as the stakes increase, because the players get better. At 50/100, an amazing player with a $500k bankroll might make about $50 / hour. I do hear that this stops being true at extremely high stakes, like $4000/$8000, where compulsive gamblers become more frequent again (relative to 50/100, the players are still far better than you'd see at a 1/2 or even a 10/20 table). But if you want to play 4000/8000 games you need a bankroll in the ballpark of $10-20M, and also there aren't that many such games. For reference, I capped out playing 2/5 NL, where I made an average of about $12 / hour. Every time I tried to move up to 5/10 I got eaten alive.
  • The hours are weird. Say goodbye to leisure time on your evening, weekends, and holidays. Expect pretty regular all-nighters, because most of your profit will come from those times when you manage to find a good table and just extract money from it for 16 hours straight.
  • It's bad for your mental health. When I was getting started, I imagined that it would be a lifestyle of pitting my mind against others, of earning money by being objectively better at poker than the other professional players. It is in fact nothing like that at all. Your money does not come from other professional players, and in fact if there are more than about 3 professional players at a table of 10, you should leave and find another table, because even if you are quite good, the professional players just don't make frequent enough or large enough mistakes that exploiting their mistakes will make you much money. No, you make your money by identifying which tables contain (in the best case) drunk tourists or (in a more typical case) compulsive gamblers pissing away money that they managed to beg, borrow, or steal in a desperate attempt to "make back their losses". It is absolutely soul sucking to realize that your lifestyle is funded by exploiting gambling addicts, and that if you find yourself at a table without any people destroying their lives it means you're at the wrong table.

In summary, -2/10 do not recommend.

There are a few things I imagine you could be saying here.

  1. Determining what you expect your future experiences to be by taking your past distribution over world models (the "prior") and your observations and using something like Bayes to integrate them is basically the correct approach. However, Kolmogorov complexity is not a practical measure to use for your prior. You should use some other prior instead.
  2. Bayesian logic is very pretty math, but it is not practical even if you have a good prior. You would get better results by using some other statistical method to refine your world model.
  3. Statistics flavored approaches are overrated and you should use [pure reason / intuition / astrology / copying the world model of successful people / something else] to build your world model
  4. World models aren't useful. You should instead learn rules for what to do in various situations that don't necessarily have anything to do with what you expect the results of your actions to be.
  5. All of these alternatives are missing all the things you find salient and focusing on weird pedantic nerd shit. The actual thing you find salient is X and you wish I and people like me would engage with it. (also, what is X? I find that this dynamic tends to lead to the most fascinating conversations once both people notice it's happening but they'll talk past each other until they do notice).

I am guessing it's either 2 or 5, but my response to you will vary a lot based on which it is and the details of your viewpoint.

The 100th anniversary of his 34th birthday

The argument is that despite some of the questionable things EA has been caught up in lately, they've saved 200 thousands lives! but did they save good lives? What have they saved really? More mouths to feed?

Yep. Some of those "mouths to feed" might end up becoming doctors and lawyers, but that's not why we saved them, and they would still be worth saving even if they all ended up living ordinary lives as farmers and fishermen and similar.

If you don't think that the lives of ordinary people are worth anything, that needless suffering and death are fine as long as they don't affect you and yours, and that you would not expect any help if the positions were flipped since they would have no moral obligation to help you... well, that's your prerogative. You can have your local community with close internal ties, and that's fine.

More cynically I think this sort of caring is just a way to whitewash your past wrongs, it's pr maximizing, spend x dollars and get the biggest number you can put next to your shady bay area tech movement that is increasingly under societies microscope given the immense power things like social networks and ai give your group.

I don't think effective altruism is particularly effective PR. Effective PR techniques are pretty well known, and they don't particularly look like "spend your PR budget on a few particular cause areas that aren't even agreed upon to be important and don't substantially help anyone with power or influence".

The funny thing is that PR maximizing would probably make effective altruism more effective than it currently is, but people in the EA community (myself included) are put off by things that look like advertising and don't actually do it.

Your explanation about how AlphaGo works is deeply counter-intuitive to me

Me too! I still don't understand why you can't just run the value model on all legal next moves, and then pick the top N among those or some equivalent heuristic. One of these days I need to sit down with KataGo (open source version of AlphaGo) and figure out exactly what's happening in the cases where the policy network predicts a different move than the top scoring move according to the value network. I have a suspicion that the difference happens specifically in cases where the value network is overestimating the win fraction of the current position due to an inability to see that far ahead in the game, and the moves chosen by the policy network expose the weakness in the current position but do not actually cause the position to be weaker (whereas the moves rated most highly by the value network will be ones where the weakness of the position is still not evident to the value network even after the move is made). That suspicion is based on approximately no evidence though. Still, having a working hypothesis going in will be helpful for coming up with experiments, during which I can notice whatever weird thing actually underlies the phenomenon.

What about more prosaic benefits like avoiding being Dutch booked? I believe that's one of the biggest benefits of consistency.

I expect that the benefits of avoiding being Dutch booked are pretty minimal in practice. If you start out with 100 A there's some cycle 100 A < 1000 B < 9 C < 99 A where some adversary can route you along that path and leave you with 99 A and the same amount of B and C that you started with, the likely result of that is that you go "haha nice" and adjust your strategy such that you don't end up in that trade cycle again. I expect that the costs of ensuring that you're immune to Dutch booking exceed the costs of occasionally being Dutch booked at some fairly minimal level of robustness. Worse is better etc etc. Note that this opinion of mine is, again, based on approximately no evidence, so take it for what it's worth (i.e. approximately nothing).

wastewater monitoring

Yeah I think this is great and jefftk and friends should receive lots of funding. Speaking of which I have just bugged him about how to throw money at NAO, because I believe they should receive lots of funding and have just realized that I have done exactly nothing about that belief.

I don't want monopolies (i.e. I think that people should be prohibited by law from colluding with other providers to increase the market prices) of goods that I buy, but for I want other people selling the same thing I sell (labor) to be forced by law to collude with me to raise the market prices.

Fair markets for thee but not for me.

This is an excellent answer. One small quibble:

Control of an aligned Superintelligent AGI is equivalent to having the keys to the lightcone, if you make it through the gauntlet of it not killing you and it listens to what you tell it, then you have the means to dominate everyone else, including others who make misaligned AGI, if yours is capable of squashing them at birth, or at the very least capable of panopticon surveillance to prevent anyone from building one in the first place.

For the record I think Yudkowsky and friends are wrong about this one. Control of the only superintelligent AGI, if that AGI is a single coherent entity, might be the keys to the lightcone, but so far it looks to me like AGI scales horizontally much better than it scales vertically.

This, if anything, makes things more dangerous rather than less, because it means there is no permanent win condition, only the deferral of the failure condition for a bit longer.

Why do discussions of white nationalism always feel the need to explicitly mention rejecting violence?

Rhymes with "Yahtzee". The last notable time white nationalists gained power did not go so well, and it is generally agreed that it did not go so well, so people with opinions that resemble that generally want to clarify that their viewpoints do not end up in that generally-agreed-to-be-bad place.

As to why the same isn't true of e.g. communists? Honestly I have no clue, but I think that indicates a problem with the communists.

Intelligence isn't caused by a few genes, but by thousands of genes that individually have a minuscule contribution but, when added up, cause >50% of existing variation in intelligence

I would bet good money that taking a genome, and then editing it until it had every gene which is correlated with higher intelligence, would not get you a baby that was even a single standard deviation above what you would naively predict based on the parents.

Consider a simple toy model, where

  1. Human intelligence is modulated by the production of a magical protein Inteliquotin (IQN), which causes brains to wrinkle.
  2. Human intelligence is a direct but nonlinear function of IQN concentration -- if IQN concentration is too low, it results in smooth brains (and thus lower intelligence), while if the concentration is too high, it interferes with metabolic processes in the neurons (and thus also results in lower intelligence). Let's say the average concentration is 1.0µg/mL.
  3. The optimal IQN concentration for inclusive fitness in the ancestral environment, and the average among the human population is 1.0µg/mL. However, the optimal concentration for intelligence specifically is 10% higher, at 1.1µg/mL (between those concentrations, improved fitness due to increased intelligence is more than offset to decreased fitness due to, let's say, "increased propensity for Oculoumbilical Tendency leading to elevated predation rates")
  4. The production of IQN is modulated by 1000 different genes IQN000 - IQN999, with the high-IQN variant of each gene occuring in 10% of the population, and where each gene independently causes IQN to increase by 0.01µg/mL.

If you have this scenario, each gene IQN000...IQN999 will explain about 0.1% of the variance in IQ, and yet using CRISPR to force just 5% more of the IQN genes to the "good" variant will lead to poorer outcomes than just leaving the system alone.

All that being said, you should be able to squeeze some results out of that technique. Just not multiple SD of improvement, at least not by doing the naive linear extrapolation thing.

So literally some takes from 5 years ago and a different account, which, if I'm correct about which name you're implying guesswho used to post as, are more saying "in practice sexual assault accusations aren't being used in every political fight, so let's maybe hold off on trying drastic solutions to that problem until it's demonstrated that your proposed cure isn't worse than the disease".

Let he who has never posted a take that some people find objectionable cast the first stone.

I assume you have some reason you think it matters that we can't use mathematics to come up with a specific objective prior probability that each model is accurate?

Edit: also, I note that I am doing a lot of internal translation of stuff like "the theory is true" into "the model makes accurate predictions of future observations" to fit into my ontology. Is this a valid translation, or is there some situation where someone might believe a true theory that would nevertheless lead them to make less accurate predictions about their future observations?