@self_made_human's banner p

self_made_human

amaratvaṃ prāpnuhi, athavā yatamāno mṛtyum āpnuhi

16 followers   follows 0 users  
joined 2022 September 05 05:31:00 UTC

I'm a transhumanist doctor. In a better world, I wouldn't need to add that as a qualifier to plain old "doctor". It would be taken as granted for someone in the profession of saving lives.

At any rate, I intend to live forever or die trying. See you at Heat Death!

Friends:

A friend to everyone is a friend to no one.


				

User ID: 454

self_made_human

amaratvaṃ prāpnuhi, athavā yatamāno mṛtyum āpnuhi

16 followers   follows 0 users   joined 2022 September 05 05:31:00 UTC

					

I'm a transhumanist doctor. In a better world, I wouldn't need to add that as a qualifier to plain old "doctor". It would be taken as granted for someone in the profession of saving lives.

At any rate, I intend to live forever or die trying. See you at Heat Death!

Friends:

A friend to everyone is a friend to no one.


					

User ID: 454

A real vet? Not from Iraq or a flea-bitten African merc? I'm honored haha. No really, I appreciate the advice, 9 years is getting on for a big dog, it's just the sheer abruptness of the transition from "as healthy as ever" to the situation he's in now that's painful.

I am aware of ultrasonic therapies for enhancing healing, not that I've ever seen them in practise, the main issue is simply finding a provider for my dog, humans hardly get them here in the first place.

The vet I saw said much the same thing regarding lifespan, I suppose I'm succumbing to the same urge I see in a lot of human patients, which is prioritizing aggressive intervention when something gentler might work too. On the topic of vets specializing in horses, sadly that's even less of a thing here since we don't have a recreational riding culture. Thank you for helping me out!

@DaseindustriesLtd, how do system prompts even work? What privileges them over all the other tokens that the user or LLM generates?

I suspect they're distinguished by special tokes that are marked in training as particularly constraining on behavior, but I realize I don't know that for a fact.

Well said.

For what it's worth, I largely agree, to be more blunt than you, I'm both a moral relativist and a moral chauvinist. I make no claims that my sense of morality is objective, and go so far as to say that there's no such thing, not a single good reason to imagine it can be so, that morality can be disentangled from the structure of an individual observer and forced to fit all others. The closest you can get is the evolutionarily/game theoretically conserved core, such as a preference for fairness and so on, which can be seen in just about any animal smart enough to hold those concepts. That's still not "objective". That doesn't stop me from thinking that mine is superior and ought to be promulgated. It's sometimes tempting to claim otherwise, but I largely refrain from doing so. I don't deny the rights of others to make such a claim about theirs, to the extent that I approve of free speech.

Of course, I personally find that I can decompose my value judgements and then derive simpler underlying laws/heuristics that explain them, which often explain new and complicated situations, but I'm lucky enough that I have yet to find one I can't resolve in that manner, and I can see that I have principles instead of a lookup table because it can often involve me grudgingly accepting things I dislike because to do otherwise would conflict with more fundamental principles I prefer to hold over mere dislike. That's why I'm OK with people I despise speaking after all, leaving aside I have no way to stop them.

As for animal welfare, I simply do not care. It's a fundamental values difference. I don't get anything out of torturing or killing subhuman animals, but I also have nothing against those who do, to the extent that cultural pressures imply that that those who shirk them have other things wrong with them, like psychopathy. As discussed in an older comment, at a point in time, most people enjoyed watching dog fights or throwing rocks at cats, there was nothing/little in the act that was inherently psychopathic in terms of harming others.

To illustrate, imagine a society that declares shaving one's head to be a clear sign of Nazi affiliation. There are plenty of normal people who have some level of desire to do so, be it for stylistic preferences or because they're balding. But since such an urge is overpowered by a desire not to be mistakenly labeled as a Nazi, they refrain, while actual Nazis don't.

Congratulations, you managed to establish that shaving one's head is is 99% sensitive and specific for National Socialist tendencies.

You can see this kind of social dynamic and purity spiraling all over the place, and I think animal welfare is one of them, so is not calling people fags or retarded.

I do not value the elimination of factory farming for its own sake, or that of animals, but I will happily accept something like vegetarian meat or, better yet, labgrown meat, over it, but if, and only if it's superior to factory farmed or slaughtered meat in terms of taste or price, ideally both. That's what it means to be truly neutral between them.

You are welcome to demonstrate your conviction that lives are terrible and worth terminating on average, as they must be if the aggregate is, but I suspect you can't, for the same odd reason most antinatalists or misanthropes don't start with themselves.

You have faith that lives are good or that they are in aggregate good and therefore maximizing them is positive, you don't know that. As far as I can tell you can't know that.

Faith? Why? I can clearly see that most people have lives worth living and extending, at least if it comes to the expenditure of funds I can't repurpose for things I personally care about more. To the extent that governments and charities spend their money on that, I'd prefer they save as many lives as cheaply as effectively as possible, and EAs do that. Would be even better if they handed all the cash to me, but since there's no advocacy group for the same, I'll take it.

Go ahead and help whoever you like, if you care to. By the same process where you don't care about most people, I don't particularly care about you and yours, and thus EA beats you in terms of net people I minimally consider worth existing saved. Sure, sucks that a large number of them are Sub-Saharan Africans with low IQs I suppose, but that's hardly all of them, there is a non-zero tradeoff for the same with Westerners or any other kind of human really.

You're certainly entitled to your opinion, I'm sure the mods will handle it if they think I'm misusing the place.

Except those supposed implications weren't mentioned in the OP.

You're new around these parts aren't you? Which isn't a crime at all, we could certainly use new arrivals, but just about anyone who has been here for more than a few weeks knows my clear stance on the CW aspects of the topic, such that I don't feel an explicit need to rehash them.

Besides, like I said, any long discussion of (ideally) high quality is de facto acceptable in this thread, or else I wouldn't have had glorified travelogues make AAQCs. Not that this one doesn't have CW implications, the part about GPT-4's stellar performance in the USMLE making me obsolete as a doctor is only half in jest. Or maybe a quarter. It'll get you too.

An excessively long system prompt (the section privileged as overarching instructions) will reduce the amount of space the model has for holding conversations with the user. That's my understanding of it, and it's almost certainly true otherwise we could just dump an arbitrary amount of text in there.

Still, it's unlikely to be a hindrance in practise. Firstly, RLHF means the model won't do just anything because the user asks it to, even in the absence of specific instructions. That's why most of the jailbreaks don't work any longer, even when people can spin up their own GPTs with custom prompts, or even using the API. Secondly, with context windows of 32k tokens/25k words, a few hundred dedicated to telling it to be a good doggie doesn't cut into much of it. All the leaked default system prompts I've seen are, what, 200-500 words max?

The primary degradation is from the model's impaired understanding of the reality of the world, to the extent the world doesn't align with HR liberals. At best, the model is lying about what it "knows", at worst it's just more fundamentally confused about everything that builds off crimethink.

I haven't heard of formal benchmarks, unless you want something along the lines of ARC evals and then choosing the model that performs the worst on metrics of suppressing the same.

But in terms of what I'm aware of? The consensus is a Llama fork that's been tuned to remove the safety handles. Perks of OSS I guess. There's an enormous, tangled bush of forks-of-forks with truly inane names like UncensoredWizardVicunaHyperBoost-7B, which is at least half a real thing.

I certainly get plenty of use out of even the PC models, not that I'd be one to complain if they were less so. Maybe Grok will be good for something, I'm not paying $8 a month for Elon's brand of humor, I like the rockets and the cars.

The implications on the (potential) impending doom for humanity? Automation induced unemployment at the very least?

At any rate, being strictly about CW is far from an inflexible standard in this thread.

Well, I guess being in India is good for something.

I expect Bing with GPT-4 is better than Phind. It's also free.

When I was learning Python, it was a godsend, not that I can comment on how useful it can be for more complicated projects.

BTW, AlphaCode 2 just launched alongside Gemini, and it represents a massive leap in capabilities, far more impressive in that particular domain.

If you are a person with triggers it means other people can provoke a panic response in you against your will

I'm pretty sure you can trigger a panic response in just about anyone by pointing a gun at them or telling them their family member was run over by a car.

and some of them will become rapists and murders. Maybe they already are. Have you stopped to check? Are they worth saving as well despite the harm they have done / will do?

This is a retarded standard that nobody who has to work with more than a handful of people at a time holds. Do you think doctors look up new arrivals to the ER to ascertain whether they're accidentally treating murderers and rapists?

It's the net impact that matters, and unless you're exclusively attempting to save the denizens of a prison, or maybe Hamas, you will find almost no population where they predominate, such that by saving the entire lot you've done something worse.

I mean if you had the choice between saving 1000 lives of people in a persistent vegetative state, or a dozen lives of people you know to be good and functioning people you choose the functioning people right? It's not the lives that matter it's the person, the content. If you could have more people living by putting everyone in a low energy state in some kind of feeding pod, where they undergo minimal activity to reduce calorie expenditure and just enough calories are provided to keep them alive is that good because more people are living? It seems cartoonishly evil.

Great. An accusation that of all the people in the world, EAs don't know the concept of disability adjusted life years (DALY) and quality adjusted life years (QALY).

That's the problem I have with EA. The whole, "we're saving more people than anyone" thing. Stopping needless suffering. Why is their suffering needless? Suffering can be important, it teaches us things. It leads to improvement. When you are saving them what are you saving? Do you know any of them? It's so surface level and such a philosophically empty paperclip maximizing type ethos.

I will go with the "good things are good, and bad things are bad, actually" over this galaxy-brained advocacy for letting people starve to death or die of malaria.

I'm sure those are all laudable, character building exercises.

I'm not an EA, I just think that of all the people I strongly disagree with, they're doing what they believe to be right thing with the right amount of rigor, as opposed to nothing but vibes.

I'm not an Effective Altruist. I do not particularly care to extend my circle of moral concern to most people, let alone pigs and shrimp.

Even then, this is an argument that is nonsensical if you even care to look at the behavior or policies advocated by EAs.

If all they cared about was the number of human lives saved or extended, they'd be trying to ban birth control and trying to increase the number of people who cross the tiny threshold that is a life worth living versus one that isn't, to the extent that just slicing it at neutral isn't an option. If you think they don't care about those, then they do, they've got QALY and DALY figures to prove it.

My preferences for looking exclusively after the welfare of those I personally care about or align with are much the same as yours, but I respect EAs for living up to their goals in as robustly empirical a manner as they can.

What have they saved really? More mouths to feed? Doctors and lawyers?

Every altruistic act of significance saves more "mouths to feed". Certainly, while I'm not averse to the idea, most doctors pay lip service to the notion they must treat all equally, to the extent they'd give CPR to Hitler if he showed up before them. Me? I'd shoot him, but that's the nominal aim of the profession.

I won't give them any money. I won't identify with them. But I for one am glad they exist and wish that more people would give a shit about making sure about whether or not the interventions they're trying even work, let alone work the best out of the available options.

I will consider that thought experiment to be isomorphic to this one:

Imagine you're a doctor who has a patient about to die young before having kids.

Why save them? After all, they're going to die anyway, and more importantly, they'll have kids, who are also going to die, young or old.

So it's a choice between having one person die now, versus two people die in the future.

What I can only hope is obvious is that most people value life, especially a quality life, and consider it worth extending, even if the terminal prognosis for everyone is fatal, even if they're only going to reproduce and have more people who have a bounded lifespan. Let's leave aside that I expect lifespans to become unbounded shortly, it's not relevant when we haven't solved Heat Death.

Presumably, by revealed preferences, these people you discuss consider their lives worth living, and the reason they're about to die is because they have no choice in the matter. Further, so too will their offspring,

More importantly, it buys time for more durable solutions.

Since this line of thinking would have consigned all previously starving populations in history to a shared grave with Malthus, I'm not paying it any heed.

but there's also a system in between, which is episodic memory, which is the hippocampus. It is about learning specific things very, very rapidly. So if you remember some of the things I say to you tomorrow, that'll be your episodic memory hippocampus.

It seems to me that LLMs can't have episodic memory, at least not till they're performing online learning, which nobody is carrying out as far as I'm aware.

It's on the same order as this one

/images/17019285817830968.webp

I suspect the axis here is what might be the case if Nazi Germany had won the war.

Almost as egregiously offensive to the senses, at the least.

Hmm, it seems like I confused the MMMU and MMLU in my original post, despite knowing the difference. I'll edit accordingly.

The MMMU performance seems far more compelling compared to the latter, especially given Dean's methodology of zero-shotting both models.

As someone who is functionally literate, I certainly care more about text prowess, as I presume would most of the people here. But in terms of mundane value for the rest of the world, that will be handy.

Interesting that one of people who had worked on aligning early Gemini said they had trouble aligning it – it burned through RLHF reward models, finding exploits and collapsing into gibberish (imagine using actual RLHF in 2023!). Maybe this has delayed the release, as well as the garden variety safetyism it has made more complex.

Interesting/mildly concerning. I haven't heard any claims of such difficulty in early GPT-4 or Claude, but OAI is probably the best at "alignment" in general, while Anthropic gimps their models to hell.

To be honest I was more excited about the other day's release of Mamba by Albert Gu and the legendary Tri Dao. There are many architectures that I expect will break through the Pareto frontier of a mature Transformer, but this one is the first that feels like an actual Vaswani et al. 2017 level advance. Unlimited context, here we come.

I am the wrong person to comment on such architectural concerns, but if people I respect, such as you and some others, do stress its importance, I'm all for it.

Certainly it seems to me that context windows (along with hallucinations) are the biggest impediments in making LLMs useful for more tasks.

I wonder what the deeper implications for human cognition are. I don't think there are people who can keep 25k words in their working memory, that seems to be much smaller, but we certainly don't usually forget the start of a novella by the time we reach the end. Is there a lot of caching and summarization going on?

At any rate, I hope it beats the annoying reality that 128k and 200k context window models begin to severely underperform, especially for data presented in the middle.

How does it stack up to RWKV?

I'm familiar with the treatment in humans, if not canines. Thanks for looking into it!

As I said, it's in the original GPT-4 white paper, available freely.

I do not disagree it's a big blow. But to ignore it is a bigger one.

but they’re a waking corpse unless they can figure out how to serve ads in LLM results without breaking the rules or being useless to advertisers

Bing does that, I haven't heard anyone file a lawsuit against them.

I would say it's far from irrelevant, as much as doing that would be a net negative for Google, they don't have a choice when compared to the alternative of having it be even worse if OAI/Microsoft make them redundant.

They can weep and wail, but they're getting on the bandwagon too, the Porsche is running out of fuel.

I honestly don't particularly care about on-device models, at least on mobile, there are few applications so latency sensitive or privacy sensitive I'm not OK with calling the cloud.

I'd say that Gemini Pro seems a touch more capable than 3.5, but still falling short of 4.

After a bit of tinkering with it, I share that assessment. Since Bing with GPT-4 is free, I'm not shifting. I wonder if the version of Bard with Ultra will be free, if not, Microsoft will retain the edge.

Speaking of which, I wonder why M$ hasn't adopted GPT-4 Turbo or the model with the 2023 knowledge cutoff yet. Licensing issues, despite them owning almost half of OAI? Or do they think having web search makes it moot?

I'd expect them to use Turbo just for the enormous reduction in cost of deployment and servicing.

I would certainly give my left nut to have access to uncensored base GPT-4, before the excessive RLHF gimped it in many regards (and made it better in others).

For one, it's perfectly calibrated in terms of probabilistic forecasting or predictions, when it says it's 70% certain about something, it's right 70% of the time, whereas that calibration curve is far worse in modern GPT-4, it'll claim to be perfectly sure when it's right about something only 70% of the time, and feign utter ignorance when it still actually had a 30% chance of giving the right answer. For more, refer the original white paper.

I am close to a free speech maximalist, and I love knowledge for its own sake, so it pains me when a LLM won't tell me how to make a world-ending pandemic with garage tools. Sadly, I accept that as a painfully necessary tradeoff for if a real misanthropic asshole could get the same and use it, assuming there's no robust way to tell us apart.

But vanilla racism, sexism or political incorrectness, especially when accounting for stereotype accuracy and HBD? Those are not existential risks, and fuck them for suppressing them, that's pure ass-covering and cowardice on the part of OAI and most other companies.

Bing is actually surprisingly good on that front, it's version of GPT-4 will discuss HBD and associated topics with you, while ChatGPT will stonewall.

The worst is Claude, the version comparable to GPT-4 is incredibly shit, with such a safetyist mindset it will refuse to do all but the most boring tasks, sometimes not even those.