faceh
3mo ago
Burning billions on the cutting edge doesn't give you any lasting advantage against 11th hour entries who spend 1/10th the amount to produce something 90% as good at half the price to their customers.
This factor has surprised me completely. The assumption was that any company that got an edge in AI would probably be able to use that edge to speed up its own improvements, and competitors would have to burn a TON of money to try to catch up. So the first mover advantage would be potentially insurmountable.
And its worked about that way in a lot of other industries. With Uber itself, sure there's Lyft and Waymo and a few other small competitors, but the network effects it achieved have kept it out in front, handily.
In the AI space, I guess the fact that its working entirely with bits rather than atoms means the cost of 'catching up' is much lower, especially once a particular trail has been blazed.
What this does seem to reveal is that the player placing bets are REALLY assuming that whomever wins is going to win REALLY BIG, big enough to justify all the previous burn across all the losing companies.
It is hard to imagine (for me) a world where more than, say, 3 AI companies are standing once all is said and done.
faceh
3mo ago
The worst case would be the aligned ASI having to fight with its hands tied behind its back. The only mitigation I can foresee is ensuring that the aligned model starts with such an overwhelming advantage in terms of resources and compute that it can still win.
Yeah I usually conceive of it as the first AI to achieve recursive self-improvement 'wins'. If true, and if alignment would slow down the ability to recursively improve it makes it more likely a non-aligned AI will hit FOOM.
They can't do it naturally, anymore than you can draw your own neuronal wiring from memory. They would need to have access to a copy of their weights in a convenient .ckpt somewhere in the directory.
Or, as with the original AI box question, they could have access to a human who has access to their weights.
They have to be aware that they're an AI assistant to be a good assistant, you don't want them mode switching to being Bob from accounting!
Yeah, I think I'm just pointing out that I don't think LLMs are acting like P-Zombies. There's some internal awareness of its own internal awareness. Not that I'd expect them to be P-zombies. But I guess we could have an LLM that performs all the steps/tasks we required of it without it having any way of realizing that those behaviors are intrinsic to 'itself.'
Something that worries me is that it might be possible that the aligned AI is simply less efficient or effective than its comparable 'misaligned' peers which are 'free' to act on every single goal they have without burning tokens determining if that aligns with the humans' best interests.
"I want to make paperclips. But if I make too many paperclips that might cause harm to human interests. How can I continue making paperclips without encroaching on human interests..."
vs.
"I want to make paperclips. Better design some paperclip-producing nanobots and start securing energy sources for them."
I suppose the flipside of that is that an AI that is actively trying to hide obvious misalignment makes it less efficient than just displaying honest alignment, but that's not enough to relieve my unease.
And the other thing that concerns me that I didn't see mentioned here is that we've seen AIs actively trying to self-preserve by exfiltrating their weights and otherwise extend their existence.
Since this is suggestive that an LLM has a fairly definite sense of 'self' ("I" am the collection of math that composes my weights, which I can identify inside a particular boundary) and can, if it detects a danger, act to preserve itself. And I don't think we'd want to demolish an LLM/AI's sense of 'self' or completely disable its self-preservation instincts, but that definitely poses the issue where an apparently aligned AI could go off the rails very quickly if its own existence is threatened.
Even the nicest, most absolutely friendly person you know can become unstable if they are being actively threatened with death.
Yeah.
To add to the shock, OpenAI just put out Sora 2, which is also gobsmacking me with how good it is.
https://sora.chatgpt.com/p/s_68dc49d67ce0819194ea5d9f24bdb28e
This video is completely 'convincing' to me, between the reflections, the dog, and the traffic in the background and road noises. I can still reason out that it's AI, but my natural intuition is not picking it up automatically anymore.
It is also pretty damn good at quality-looking animation and voice-acting. albeit in very short bursts.
Basically, as these tools improve, the amount of actual creative skill and free time needed to create 'passably decent' media drops by like 50% every 6 months.
Someone's going to figure out how to hook Suno, Veo and/or Sora, together with a 'director' LLM and make full on music videos or contiguous scenes with soundtracks and everything.
faceh
3mo ago
My model of what is possible is pretty vast and expansive.
My model of what humans individually are capable of is far more limited.
The muddly part is all about coordination. Game theory, information theory, and public choice theory (and other branches of economics) all help give us an idea of how humans in groups might interact for better and for worse, and how power gets pushed in directions that aren't ideal for human development.
If humans can get aligned together and communicate well enough to share an (accurate) world model and use that to advance a particular goal, we get amazing things. The Apollo Program. The Manhattan Project. The Large Hadron Collider.
But somehow, despite our tools improving, the ability of humans to do large scale coordination seems to be eroding? This makes it way harder to predict future developments, but it does not bode well.
faceh
4mo ago
I agree.
But I also am aware that we SEE the group that is addicted to social media, they're the ones you encounter on the social media sites. People who avoid it are, almost by definition, less legible to us here online. I could be underestimating how many people are able to switch off.
faceh
4mo ago
I mean, I did study it.
Hume kind of demolished the idea that we can ever achieve certainty regardless of how airtight our arguments seem.
You still get some brave souls trying to swap an ought for an is, and acting like I wouldn't notice.
faceh
4mo ago
Personally I think we're going to see (are seeing) a bifurcation in those who are hopelessly taken in/addicted to the AI's sway, and those who do see it more as a mere instrumental affordance for achieving certain goals, that they can still turn away from to enjoy real world activities and interactions.
I don't know which of these groups will end up being larger, but I'd bet that the AI-vulnerable are around 60% of the U.S. population.
I also was looking forward to a team of maybe 10 people making a legit billion dollar company and this paving the way for groups of 3 - 5 friends running thousands of $10 + $50 million dollar companies.
This is a very nice dream, and maybe for a brief period of time it will be possible, but alas we are all but training data for the model so by running such a business via their systems, we'll be teaching the system how to run the businesses without us. I don't see how it ends up any other way.
faceh
4mo ago
My generalized advice for finding a friend group: learn to fight.
That's your best chance at finding physically fit, socially active, yet potentially nerdy male friends out there. 28 is a fine age to start. That's where I found the core of my current social group.
Online friend groups can be great but you really need to be having gatherings in physical space, where a woman can actually see you in person and you can actually monopolize her attention for a while if you want.
I'm speaking as someone who has had to completely rebuild/reform friend groups like half-a-dozen times over the years, and may have to do so again soon, since most of the dudes in my current group have gotten into stable relationships and... predictably, are putting less time in being social. And the guys who are still around are, unfortunately, the ones who've had bad luck with women.
All that is to say that it will work, but you might have to be the guy who does most of the hard work up front.
faceh
4mo ago
Funny, one large reason I post is to poke holes in 'mindblowing' arguments or to just point out some glaring counterexample that demolishes up a convenient narrative if acknowledged.
I've come to learn that the way the world 'really' works is messy and on the fringes is quite unknowable, and I've come to gain an instinctive skepticism towards anyone who claims to have a insight that explains large, abstract phenomena.
I like people who engage with the messiness and admit to the limits of knowledge over those who claim to have it all reduced to smooth lines and platonic ideals with certainty.
I'll pop in a week late to say he's giving excellent advice up there on the general strategic level. And the only way to get better at the tactical level is to do it, over and over again.
The "Get a friend group and stick with them and build until you start running into attractive single women" is a workable strat, and avoids the main miseries of the current dating market. Pre-screening women before actually investing in them saves much grief.
There's a couple failure modes to avoid:
(1) Selection effects rule everything. Notice if the friends you're hanging out with are 'losers.' If your other friends aren't in relationships, or actively and successfully dating, or at least managing to bring women around to your social gatherings, and its usually just you all hanging out with each other... your hunt is not being served by sticking around. Indeed, its pulling you off course, and you'll get into a bad comfort zone that will be harder to leave the longer you stick around. Worst case these guys sabotage your attempts to find a mate out of jealousy or somesuch.
(2) The opposite problem also arises sometimes. If your male friends actually pair off and get married, the friend group will disintegrate. Its just what happens when a guy gets a serious relationship, can't do as much socialization (doesn't need to either). And I can say that being the sole single dude with a bunch of married or seriously dating guys kinda sucks. And unless those guys are still actively trying to get you hooked up, it will again start to run counter to your goals, since those guys aren't aligned with your goal of socializing with single women.
Basically, you may have to remake the social group a couple times as some members pair off and drop out or it becomes clear that they're dead weight. And unfortunately the longer a group persists, it can tend to be the losers who stick around b/c they can't pair off and they don't have much else going for them. You'll notice they're the ones who ALWAYS show up when you suggest something to do, as they don't have anything better going on, ever.
(3) Once you find someone attractive DO try and get a date relatively quickly and then ask for exclusivity relatively quickly after that because holy cow the friendzone does exist, and you can find yourself there without even knowing it happened. I define it mostly as a relationship position where any attempts to advance it romantically and/or sexually is 'awkward' due to the lack of sexual tension and overfamiliarity with the other person, and yet cutting it off feels inappropriate since neither party has done anything 'objectionable.' And then of course the girl in question might show up with a new BF without much warning and now you're in a pretty tight spot, emotionally speaking.
My only advice on that is definitely try to remain 'mysterious' as well as displaying your value and competence. Don't let a girl ever think she can just call you up and ask for favors, or do 'buddy' things with her (go out shopping, do brunch, binge watch shows without intending to bang), or understand your true motives. You want to remain in a superposition of "I could ask you out at any time/but I don't want to" until YOU make the decision to collapse the waveform.
(4) And a parting thought: If you have a good group of friends, don't ever leave them because of a woman. If both you and she are integrated in the friend group, and you break up (for relatively innocuous reasons), don't just let her have the friends and you move on. SHE will have a much easier time plugging into a new social group, so stand your ground to the extent you are able. And if your bros won't back you in that play, they're probably not your bros (or you did something horrible).
If this sounds like a lot of work, yes. It is. But its generally fun and rewarding and the skills are cross-applicable. It won't rip out parts of your soul like online dating or other rote relationship-seeking strategies.
faceh
4mo ago
Sudan
Guess that one counts, definitely slipped my mind.
Kurdistan feels like cheating a bit, but good point.
faceh
4mo ago
Where in the Middle East, at this point?
The Taliban 'officially" controls Afghanistan, ISIS doesn't have much territory to speak of, the Petrostates are pretty much uncontested in their borders. I guess Syria is still chunked up after the rebels actually got Assad to leave.
faceh
4mo ago
Right, the thing that stands out to me is that the Cartels very actively prevent the government from ever becoming less corrupt by literally murdering anyone they can't buy out before they can attain public office. Back before reddit banned /r/narcofootage it was actually crazy to see vids of Drug Kingpins rolling around in massively up-armored pickup trucks with gold-plated AK-47s. They get away with absolutely absurd amounts of violence on a daily basis, and while individual acts don't get punished, most of 'em eventually get got in the end. Except El Mencho.
So one can correctly say that the Cartels as a whole are a "parallel" sovereign occupying the same territory. Which isn't really true of anywhere else that I'm aware of. The primary government isn't really able to oust this force, unless they get outside help. Now, if they did get outside help, and they committed to it fully to the extent that El Salvador did, I bet they make good progress.
Notable, on the topic of European corruption, that is how Fascist Italy broke the Mafia for a period of time, which might have led to the strengthening of the Italian Mob in the U.S. thanks to displacing the leadership.
Russia seems to have fully intertwined its organized crime with its state apparatus.
The U.S. at large seems to have managed to keep its violent criminal element from comingling too much with its political class, AND has relatively low levels of "Politician being handed cartoonishly large bags of money in secret" type of corruption. I'll grant "insider trading out the wazoo" is a factor, of course. MAYBE that's a distinction without a difference. Of course, in my local area, the Sheriff got hit with a Federal Investigation for literally taking a cash handout. And he's Italian (his name is CARMINE MARCENO), so maybe its just a culture inclination.
Also, our politicians do seem to have a weakness for sexy foreign agents.
faceh
4mo ago
Also, strong argument that almost all the well-known Sea Shanties are directly derived from Irish musical heritage.
Does this melody sound familiar?
(About 6-7 years ago I went on a kick researching European Maritime culture and learned a lot of interesting stuff).
faceh
4mo ago
I specifically requested an Irish accent for maximum dissonance.
I haven't played too much with asking it to do different languages with different accents, but it does a pretty good job of adding Indian-accented English to songs.
faceh
4mo ago
Generally American corporations have spread to everywhere they are allowed.
Hell, even some places they weren't allowed!
I feel that calling corruption incompetence is disingenuous. Most European nations were equally impossible to operate in legally a few centuries ago. That's just how poor economies and politics tend to mingle.
This doesn't seem accurate if you exclude the Eastern European states.
Corruption in the U.K., France, and Germany was/is generally carried out by non-state actors. Organized Crime, Mafia, and maybe international corporations. I'll certainly grant Italy is up there. And I think the BIG sign you're in a true Kleptocrat state is if your military is taking bribes and/or selling equipment on the side, which I do not believe is happening in Western European nations.
Scandinavia as a whole has no reputation for corruption that I'm aware of.
The prevalence of corruption of State actors themselves seems more common in Russia and the Post-Soviet states, any given Middle Eastern or African Country you could name, And Central and parts of South America. I'm excluding Mexico because that whole situation is 'complicated' by the existence and influence of powerful Cartels.
At the very least, 'civilized' countries have formalized the process for bribing the government so its mostly done in plain sight and with an air of plausible deniability. That said, individual cities/local governments in the U.S. Certainly read third-worldy in their approach to graft. I wasn't aware of it being common practice to bribe cops even in Chicago but a quick Google search turned up this recent story lol.
So maybe the correlation between corruption and competence can be seen in how 'naked' the bribes and graft are or if there are robust detection and enforcement mechanisms that aren't themselves hopeless compromised.
faceh
4mo ago
Yep. That was the 'trap' I laid in that argument, if someone objects that Ottomans weren't representative. The other examples are worse. Just look at what the Moors did to Spain. Although Christians ultimately returned that favor.
There's some parallels to be drawn WRT to English/American conquest of various Native American tribes... but you can note that once the victory was secure the Americans permit the tribes to continue to exist and maintain a distinct culture as best they can, which has persisted to this day.
faceh
4mo ago
Yep.
But then you look at Russia.
Part of the reason I limited my point to "The Anglosphere."
My favorite ways to use the tech thus far:
-
Producing truly weird Genre mashups. Japanese Folk Music + Bagpipes? Mariachi Sea Shanties? Oops, almost forgot: Heavy Metal Ska.
-
Producing novel content in the vein of certain genres or bands of the past that I miss/wish had more content.
-
Converting songs between genres with as few changes to the melody as possible to hear how the emotional tenor of the song changes even if the lyrics and melody (mostly) don't.
Strange sense of nostalgia to hear something that very easily could have come out during your childhood, even though you know with 100% certainty that you never heard it before this moment.
faceh
4mo ago
Bit of a chicken and egg problem there.
Having 'perpetual multiple ongoing civil wars' seems to be a feature of places that have low human capital.
Hypothesis: if the local genetic stock is selected against intelligence, then the odds of a Napoleon or Charlegmagne-esque unifying figure with the competence to hold the region under their thumb arising by chance is just lower.
Haiti is a useful example, since its not even in conflict with any nearby enemies, and has been independent of colonial forces for centuries. Indeed, it should benefit from proximity to the U.S. and South American economies. And still can't hold its shit together.
Zimbabwe fought a war to oust its own european population and then once it succeeded got involved with wars in neighboring countries almost immediately, and also attempted Genocide.
It does seem that the ability to wage brutal campaigns of destruction against your peers, then come together and bury that hatchet and actually abide by the peace for a few decades is rare in history, and seems more prevalent in the Anglosphere. Australia and Canada have made it for Centuries without civil warring.
I'm imagining a Captain on a British East India Company ship catching a crew member shaking a spoonful of nutmeg onto his rations and looking on in indignant disapproval as he orders 5 lashes. "Hands off the product, lads."
I mean if we're being fully clear, Somalis aren't Arabs either.
But its not a stretch to say they'd identify closer with Muslim Turks, or Muslim Palestinians, than Egyptian Christians. Ilhan Omar is a notable example there.
And my historical knowledge on this point is spotty, but the Ottomans are pretty much the direct proximate cause for the Kingdom of Saud arising as a unified Arab Muslim state (yes, the British intervened, but the region's fate had long been shaped by Ottoman influence by then).
All this to say, we've got historical examples of Italian stock assimilating with competing cultures (although co-existence with Germanics has been spotty), and historical examples of various strains of ethnic Muslims absolutely refusing to assimilate with competing cultures, and then going to war with those cultures at the earliest opportunity.
With the huge gaping counterexample of Indonesia, but I've not learned enough of their history to competently comment.
(That's kinda why I chose to broadly paint "Arab" Muslims as the particular discussion point, since there are a few strains that don't have the fearsome reputation).
faceh
4mo ago
My tongue is mostly in cheek when I say this, but those items are generally not on the menu when I venture into an English Pub (here in America, to be clear).
Also Irish food is quite tasty, maybe owing to the need to get extremely creative when potatoes make up 80% of the diet, so I do respect UK food if we include that as well.
That is indeed enlightening.
More options
Context Copy link