SnapDragon
2mo ago
Citation needed...? It's a little hard to ask the pig. And even if true, should I care overmuch that the pig "feels stressed" for the last hour of its life? Humans go through worse (to say nothing of how animals die in nature!). If you want me to care about animal welfare, you should focus on the part that really matters - the life the pig lived - rather than the lurid, but ultimately unimportant, details of its death.
SnapDragon
2mo ago
"Brutally" slaughtering a pig in "disgusting" "industrial" conditions? Those are very subjective words. The pig doesn't care that it's not being given a dignified sendoff by its loving family at the end of a fulfilled life in a beautiful grassy glade with dandelions wafting in the breeze. Humans fear death; animals don't even understand the concept. As long as we kill them quickly, I really don't give a shit how it's done.
Which isn't to say I don't have concerns about factory farming. The rest of the pig's life may be filled with suffering, and (IMO) we're rich enough, as a society, to do better. My morality-o-meter is ok with sacrificing, say, 0.01% of value to humans to improve the life of pigs by 500%.
SnapDragon
2mo ago
Maybe I'm missing some brilliant research out there, but my impression is we scientifically understand what "pain" actually is about as well as we understand what "consciousness" actually is. If you run a client app and it tries and fails to contact a server, is that "pain"? If you give an LLM some text that makes very little sense so it outputs gibberish, is it feeling "pain"? Seems like you could potentially draw out a spectrum of frustrated complex systems that includes silly examples like those all the way up to mosquitos, shrimp, octopuses, cattle, pigs, and humans.
It'd be nice if we could figure out a reasonable compromise for how "complex" a brain needs to be before its pain matters. It really seems like shrimp or insects should fall below that line. But it's like abortion limits - you should pick SOME value in the middle somewhere (it's ridiculous to go all the way to the extremes), but that doesn't mean it's the only correct moral choice.
SnapDragon
2mo ago
I agree, when I worked at Google I remember their security measures being extremely well-thought-out - so much better than the lax approach most tech companies take. However, I DON'T trust their ideological capture. They won't abuse people's information by accident, but I will not be surprised if they start doing it on purpose to their outgroup. And they have the tools to do it en masse.
SnapDragon
2mo ago
On most forums, if you're a bad actor waging the culture war, it's probably a decent strategy to post a bunch of links like this that are ridiculous non-sequiturs. Most people are too lazy to follow them and have the (usually reasonable) assumption that what's said in them is being accurately represented. Fortunately, I think The Motte is better than that. Looking forward to guesswho's inevitable (re-)permabanning. We need good leftist posters, but he's not one.
SnapDragon
3mo ago
I'm far from an expert (and I doubt anyone else in this thread is either), but I'm not sure I really agree with your "extremely dangerous" assessment. Lots of things have a 100% kill rate. Like, congratulations, they've reinvented rabies? A virus that represents a serious risk to society needs to combine a number of unlikely factors, and "killing the host" is probably the easy part. (Ironically, after a certain point, high lethality makes a virus less threatening - a virus's host needs to survive to spread it on!) To truly threaten civilization, you'd have to combine it with a long asymptomatic but highly contagious incubation period.
Of course, because the media are idiots, the article you linked mentions the "surprisingly rapid" death of the mice as if that's supposed to make it more, not less, scary. Ah, journalists, never change.
SnapDragon
3mo ago
Ugh, what a ridiculous take. The ability to move a body and process senses and learn behaviour that generates food is miraculous, yes. We can't build machines that come close to this yet. It's amazing that birds can do it! And humans! And cats, dogs, pigs, mice, ants, mosquitos, and 80 million other species too. Gosh, wow, I'm so agog at the numinous wondrousness of nature.
That doesn't make it intelligence. Humans are special. Intelligence is special. Until transformers and LLMs, every single story, coherent conversation, and, yes, Advent of Code solution was the creation of a human being. Even if all development stops here, even if LLMs never get smarter and these chatbots continue to have weird failure modes for you to sneer at, something fundamental has changed in the world.
Do you think you're being super deep by redefining intelligence as "doing what birds can do?" I'd expect that from a stoner, not from a long-standing mottizen. Words MEAN things, you know. If you'd rather change your vocabulary than your mind, I don't think we have anything more to discuss.
SnapDragon
3mo ago
SnapDragon
3mo ago
Wow, you're really doubling down on that link to a video of a bird fishing with bread. And in your mind, this is somehow comparable to holding a complex conversation and solving Advent of Code problems. I honestly don't know what to say to that.
Really, the only metric that I need is that ChatGPT makes me more productive in my job and personal projects. If you think that's "unreasonably low", well, I hope that our eventual AI Overlords can hope to meet your stringent requirements. The rest of the human race won't care.
Then I tried it on Day 7 (adjusting the prompt slightly and letting it just use Code Interpreter on its own). It figured out what it was doing wrong on Part 1 and got it on the second try. Then it did proceed to try a bunch of different things (including some diagnostic output!) and spin and fail on Part 2 without ever finding its bug. Still, this is better than your result, and the things it was trying sure look like "debugging" to me. More evidence that it could do better with different prompting and the right environment.
EDIT: Heh, I added a bit more to the transcript, prodding ChatGPT to see if we could debug together. It produced some test cases to try, but failed pretty hilariously at analyzing the test cases manually. It weakens my argument a bit, but it's interesting enough to include anyway.
SnapDragon
3mo ago
So, I gave this a bit of a try myself on Day 3, which ChatGPT failed in your test and on Youtube. While I appreciate that you framed this as a scientific experiment with unvarying prompts and strict objective rules, you're handicapping it compared to a human who has more freedom to play around. Given this, I think your conclusions that it can't debug are a bit too strong.
I wanted to give it more of the flexibility of a human programmer solving AoC, so I made it clear up front that it should brainstorm (I used the magic "think step by step" phrase) and iterate, only using me to try to submit solutions to the site. Then I followed its instructions as it tried to solve the tasks. This is subjective and still pretty awkward, and there was confusion over whether it or I should be running the code; I'm sure there's a better way to give it the proper AoC solving experience. But it was good enough for one test. :) I'd call it a partial success: it thought through possible issues and figured out the two things it was doing wrong on Day 3 Part 1, and got the correct answer on the third try (and then got Part 2 with no issues). The failure, though, is that it never seemed to realize it could use the example in the problem statement to help debug its solution (and I didn't tell it).
Anyway, the transcript's here, if you want to see ChatGPT4 troubleshooting its solution. It didn't use debug output, but it did "think" (whatever that means) about possible mistakes it might have made and alter its code to fix those mistakes, eventually getting it right. That sure seems like debugging to me.
Remember, it's actually kind of difficult to pin down GPT4's capabilities. There are two reasons it might not be using debug output like you want: a) it's incapable, or b) you're not prompting it right. LLMs are strange, fickle beasts.
SnapDragon
3mo ago
I'm on Apple's AI/ML team, but I can't really go into details.
SnapDragon
3mo ago
...are you seriously asking this? I'm not an insect. If you want to claim some observation of insect behavior has even the slightest relevance to human society, the burden of proof's on you.
SnapDragon
3mo ago
Hi, bullish ML developer here, who is very familiar with what's going on "under the hood". Maybe try not calling the many, many people who disagree with you idiots? It certainly does not "suck at following all but the simplest of instructions", unless you've raised this subjective metric so high that much of the human race would fail your criterion. And while I agree that the hallucination problem is fundamental to the architecture, it has nothing to do with GPT4's reasoning capabilities or lack thereof. If you actually had a "deep understanding" of what's going on under the hood, you'd be aware of this. It's because GPT4 (the model) and ChatGPT (the intelligent oracle it's trying to predict) are distinct entities which do not match perfectly. GPT4 might reasonably guess that ChatGPT would start a response with "the answer is..." even if GPT4 itself doesn't know the answer ... and then the algorithm picks the next word from GPT4's probability distribution anyway, causing a hallucination. Tuning can help reduce the disparity between these entities, but it seems unlikely that we'll ever get it to work perfectly. A new idea will be needed (like, perhaps, an algorithm that does a directed search on response phrases rather than greedily picking unchangeable words one by one).
To be honest, it sounds like you don't have much experience with ChatGPT4 yourself, and think that the amusing failures you read about on blogs (selected because they are amusing) are representative. Let me try to push back on your selection bias with some fairly typical conversations I've had with it (asking for coding help): 1, 2. These aren't selected to be amusing; ChatGPT4 doesn't get everything right, nor does it fail spectacularly. But it does keep up its end of a detailed, unprecedented conversation with no trouble at all.
SnapDragon
4mo ago
Lockdowns aren't on the pareto frontier of policy options for even diseases significantly deadlier than covid imo, just because rapid development and distribution of technological solutions is possible, but ... covid killed one million people in the united states. Yes, mostly old people, but we're talking about protecting old people here. No reason to pretend otherwise.
Speaking of government policy, I wonder how many lives were lost because we couldn't conduct challenge trials on COVID? It was almost the ideal case - a disease with a rapidly-developed, experimental new vaccine and a large cohort of people (anyone under 40) for which it wasn't threatening. If we were a serious society - genuinely trying to optimize lives saved, rather than performatively closing churches and masking toddlers - I wonder how early we could have rolled out RNA vaccines for the elderly?
SnapDragon
4mo ago
Heh, yeah, good example. I happily commit atrocities in videogames all the time. I hope there will continue to be an obvious, bright-line distinction between entities made for our amusement and entities with sentience!
SnapDragon
4mo ago
Maybe shot 5 times? Or maybe 32 times? I suppose there's not much difference between the two.
SnapDragon
4mo ago
I mostly agree with you, but I want to push back on your hyperbole.
First, I don't think doing RLHF on an LLM is anything like torture (an LLM doesn't have any kind of conscious mind, let alone the ability to feel pain, frustration, or boredom). I think you're probably not being serious when you say that, but the problem is there's a legitimate risk that at some point we WILL start committing AI atrocities (inflicting suffering on a model for a subjective eternity) without even knowing it. There may even be some people/companies who end up committing atrocities intentionally, because not everyone agrees that digital sentience has moral worth. Let's not muddy the waters by calling a thing we dislike (i.e. censorship) "torture".
Second, we should not wish a "I have no mouth and I must scream" outcome on anybody - and I really do mean anybody. Hitler himself doesn't come close to deserving a fate like that. It's (literally) unimaginable how much suffering someone could be subjected to in a sufficiently advanced technological future. It doesn't require Roko's Basilisk or even a rogue AI. What societal protections will we have in place to protect people if/when technology gets to the point where minds can be manipulated like code?
Sigh. And part of the problem is that this all sounds too much like sci-fi for anyone to take it seriously right now. Even I feel a little silly saying it. I just hope it keeps sounding silly throughout my lifetime.
SnapDragon
4mo ago
How so? He could have just retired slightly sooner, still quite rich and still doing the rounds on news/talk shows answering tough questions like "how does it feel to have saved eleventy-trillion lives with Science(tm)?" Rand Paul constantly grilling him would barely even be reported on, let alone actually affect his life.
SnapDragon
5mo ago
Not only do you get to use "think of the children", you also get to partake in socially-approved hate for a group of weirdos for their innate characteristics. Humans have always had an appetite for doing this, but in modern times there are far fewer acceptable targets.
SnapDragon
5mo ago
Interesting. I admit ignorance here - I just assumed any UK-based newspaper would be very far to the left. (The video itself still seemed pretty biased to me.) Thanks for the correction.
I happen to believe that most of what our modern legislators do IS net-negative
If you think all regulations can be monolithically grouped into a giant "is bad" category...
If the difference between "bad regulations" and "good regulations" don't matter...
I have no idea how you're extracting these arguments from what I said. (shrug)
Comments like these are pretentious and unnecessary.
For the record, I wasn't trying to be mean-spirited (just "funny"), but I see it could come off that way.
SnapDragon
5mo ago
Cool, cool. So, the obvious follow-up question is, can we just keep those critical federal employees, and drop everyone else? We might even survive firing the seven critical workers who were kept off furlough to keep people away from the Washington Monument.
I'm being a little facetious. You have a point, of course - lots of government services seem extraneous right up until the point where you (or someone else in a worse situation) desperately need them. It would be great if there was an option somewhere between 0% and 100% of our current government, where the first 10% to go isn't the part calculated to maximize spite.
SnapDragon
5mo ago
Sorry, it sounds like you want some easy slam-dunk argument against some sort of cartoonish capital-L Libertarian, but that's not who you're speaking to. :) I don't want NO government and NO regulations - of course some regulations are good. But that says nothing about whether we have TOO MUCH government and TOO MUCH regulation right now. Most of the important obviously good stuff has been in the system for decades (if not centuries), because it's, well, important. And even if we kicked legislators out for 51 weeks out of every 52, the important stuff would still pass because it's, well, important. I happen to believe that most of what our modern legislators do IS net-negative, and I'm afraid you can't just hand-wave that away with a strawman argument.
As for YIMBYs, bless your heart Charlie Brown, you keep trying to kick that football. Surely one day they'll win! You yourself linked an article about the dire straits we're in. "Don't try to stop or slow down the government, we need it to fix all the problems caused by the last 50 years of government!"
And that's the point. That's the one, last, important step that (much of) science is lacking. Have you built something that works AT ALL? It's not that engineering doesn't suck. It's that modern "science" is even worse, because so much of its product (random unreplicated research papers, written on esoteric subjects, skimmed by friendly peer reviewers and read by nobody else) never needs to pass that final filter.
More options
Context Copy link