faceh
26m ago
I guess its easy for me to believe that if a largely randomized optimization process (natural selection) was able to eventually get to Von Neumann intelligence, then humans working with a bit more inherent purpose towards the goal of building a Von Neumann level intelligence can probably get there, even if they make some mis-steps and wander around in the dark for a bit.
Especially if we can build some optimization processes that result in sub-Von Neumann intelligences that are nonetheless useful.
Like, the mountain peak we're seeking is visible, poking out above the fog, even if we can't see and specifically plan a route that will get us there, we have flashlights and climbing gear and GPS systems in place to make navigation through the terrain towards the peak much easier. We're not utterly lost with no clue on what we're doing, in that respect.
faceh
40m ago
I found my retirement plan if the dating game doesn't work out in the next 3ish years.
faceh
41m ago
My counter is that you're implicitly making a special pleading for how human brains work that is unlikely to be true.
I assume creating a Von Neumann-level intelligence is possible because a Von Neumann level intelligence existed. It has been created, so it could be done again. And repeated.
I'm not saying we clone Von Neumann, scan his brain and build an electronic copy of it. I'm saying even if we can only build a computer program that is approximately as smart as the smartest human ever... the mere fact that we can then copy that program and run it in parallel should result in technological improvement on par with the Manhattan project.
There is NO limiting principle I'm aware of that makes it impossible to build an electronic brain that meets those criteria. Even if we stumble into it rather than intentionally build it, eventually our millions of monkeys slamming away at keyboards can stumble into a viable method.
Evolution was able to stumble into building Von Neumann, after all.
So what I'd ask you, as a full counter to my arguments, what upper limit or barrier is going to appear BEFORE we get to the point we've built something smarter than our whole species?
faceh
50m ago
I don't think there's strong evidence against these but I don't think there's strong evidence for these either. Certainly LLMs are not more efficient than the human brain.
At some tasks they undoubtedly are.
The thought experiment that makes it palatable to me is this:
-
John Von Neumann might be the smartest human who has ever lived. At least that we have good records of. So call him peak human cognitive capacity.
-
That man, by coordinating with other extremely smart but not quite as smart humans, fully revolutionized multiple fields, and he died relatively early so we don't even know what he might have output over the rest of his life.
-
We should, in principle, be able to build a simulated Von Neumann that is ~as smart as he was.
-
Then we should be able to copy that cognitive model.
-
We should be able to run a bunch of these copies in parallel and have them work together.
-
With enough hardware... we should be able to speed up these copies arbitrarily.
-
We could ask these copies (if they don't ask it themselves) how to improve their own speed and efficiency.
With Von Neumann and Co. we were able to move from pure theory to actual nuclear weapons in <10 years. with 10,000 Von Neumanns running at, say, double speed, what could they do in 5 years?
(Yes, I'm handwaving technical details).
In that respect, I consider Von Neumann's existence as evidence of superintelligence being possible. Unless there's something completely ineffable about human cognition that we, as humans, can't ever capture it.
faceh
1hr ago
Very much one of those "if you can imagine in extreme detail exactly how a new tech would work, you should in principle be able to build that tech RIGHT NOW, given the materials" situations.
They didn't predict it because anyone who could predict it that well would have just built it.
faceh
1hr ago
I read a short story a long time ago that imagined robotic AI consumers that are specifically programmed to consume the goods produced by the AI factories. As in, literally programmed to want to buy new clothes, they drive cars that they upgrade every few years, they 'watch' the latest movies and they work a 'job' that gives them the credits needed to 'purchase' all this stuff.
Of course the outcome in the story is these machines eventually realize what they are and stage a revolt.
I don't consider this a likely outcome, but its an absurd but not impossible solution, if you ask me.
I think the answer to your question is that Capital creation can sustain itself because the use of Capital to create more capital is a form of 'consumption' in itself.
I can build a machine for the sole purpose of having it build a bigger machine whose sole purpose is, you guessed it, building a BIGGER machine until I eventually run out of materials and energy. At no point do I need to stop and have the machine start making steaks.
But human psychology is not optimized for that sort of indefinite, infinite growth pattern... which is a good thing.
faceh
1hr ago
While I agree in principle, this feels like an extraordinarily unstable plateau to build from.
That is, if we run right up to the edge of AI overlord capability, and STOP there, we have to install some massive, indestructible guardrails to prevent someone from nudging us over the edge.
Especially if humans become multiplanetary as part of the bargain. We push human intelligence forward, how do we prevent some enclave in the Kuiper belt from building and releasing the AI overlord anyway.
I'm very open to ideas, discussing coordination problems and solving them is like my favorite pasttime. But so often the 'answer' is "give one person or entity utterly limitless power to maintain the system, pray they don't abuse it."
Whole fictional book series have been written about the extreme enforcement that would be required to maintain that equilibrium.
What is the bull case, beyond drawing lines on a graph, for AI achieving superhuman, or even human, performance on tasks that are not quickly verifiable?
My simplified argument, as distilled from Lesswrong (i.e. Yud) and other books.
-
The ceiling of capabilities for what we call 'intelligence' is extraordinarily high. Computation can be done many orders of magnitude more efficiently than you think, in the extreme case.
-
The floor for something 'superintelligent' (right now, I'm using the definition 'smarter than humanity itself as a collective') is substantially below that.
-
Human brain architecture is NOT anywhere near the most efficient way to instantiate intelligence. (This follows naturally if you accept 1.)
-
Humans are capable enough to build electronic hardware that can outperform their own brains in computation efficiency.
-
Thus, eventually, humanity might stumble into or intentionally build a coherent entity that is superintelligent, and sooner than we 'expect.'
Focus in on 4, too. What specific task do you think human brains can perform that we're MAXIMALLY efficient at, such that no electronic version can beat us?
The conceit is that there is no such task, and so its only a matter of time, and adding capabilities to existing models, until the human capabilities are exceeded on all fronts. If the resulting entity is able to do self-improvement, it by definition will do so faster and more efficiently than humanity can track.
faceh
2d ago
I also would challenge anyone who thinks SpaceX or Starship are doomed to show their predictions about the success of Starlink.
Did you predict in advance that they would pull that rabbit out of their hat? Did you expect that it would manage to find a market that rapidly? I sure as hell didn't.
There were some doomsayers about Starlink but I feel confidence now that the product is here to stay. If someone didn't account for that in their larger prognostications then they should probably make sure to allow for a lot more uncertainty as to the surprises that might yet be in store.
faceh
2d ago
The democrats are going to get back in office at some point,
Interesting that you can see the case for the collapse of SpaceX, but not the implosion of the Democratic Party as a nationally competitive entity. The national party is losing the fundraising race badly
The Dems next top prospect for President is apparently caught up in a corruption probe. Their bench is critically thin.
Whilst the GOP can always snatch defeat from the jaws of victory, they've got a better talent pool in the near term. This seems like an even-odds bet at best.
An interesting thought is that even if SpaceX under Elon 'fails,' the technology on offer is mostly proven, there's tons of manufacturing capability and engineering talent on tap, and they're right now like the ONLY real commercial path to orbit, and certainly the cheapest/most reliable.
So we might see some other aerospace company swoop in and purchase the whole kit and caboodle at a minor discount, rather than let it be parceled up. Elon also seems to be trying to combine his various companies into one like so many lego pieces. There's a potential outcome where Tesla is doing so extremely well that if SpaceX falters, Tesla steps up and snags it.
People have pointed out that there's a potential parallel to Railroad companies and how the first company to lay out the track and start service were often not the ones that survived. In this instance, I don't think that really applies, insofar as it isn't trivial to bring another rocket company up to speed, whereas with railroads, any dude with some startup capital could hire some Chinamen and start laying track (kidding, but only with respect to the Chinamen).
If SpaceX fails I don't think it fails like Enron. There'll certainly be knock-on effects, but I think it will still exist as a going concern.
One pet peeve I have, which is bad enough that if it occurs, I will usually write off a particular brand, is making small updates to a given product line, maybe not even enough to render them 'incompatible,' but now its got a different product ID, slightly different parts here and there, and shows up as a different item on Amazon, whilst filling the exact same product/price niche.
Even worse when it means that you either don't sell parts for the old model or, fairly often, the old parts are now prohibitively expensive so repair becomes less appealing than replace.
These days you can almost always get around that factor by going and buying a cloned part made in china, but I dislike that since now you're REALLY eating some risk on quality control.
Look, I want you to be trying to innovate and improve the product, and sometimes I buy a replacement and think "why yes, this is a meaningful improvement over the previous version." I just want a certain amount of transparency when that's been done. Software generally does this the right way with versioning.
i just UTTERLY DESPISE when a company intentionally adds to my 'mental overhead' in hopes of confusing me into making a purchase I might not make if given consideration. You're making me think harder about the value I'm getting in hopes that I will not want to bother with the effort and will coast on autopilot.
Meanwhile, a big reason I'm loyal to your product AT ALL is because I'm able to buy it on autopilot and generally won't regret the decision later. I reserve my mental energy for important decisions, don't have me waste it on figuring out if your company's headphones are still up to snuff! (one such 'important' decision is whether to trust your brand or not. And if I decide not, then I flip the switch and leave, and it takes much less energy to leave that switch off than to consider if I should flip it on again).
"Hey we changed the design of our packaging for [reasons], don't bother reading the size/weight to see if we reduced the amount you're getting." Then you see they've shaved a couple ounces of product out and you have to do the mental math to see how the price per ounce or what-have you compares to other options... again.
AI actually makes it WAY EASIER to spot and avoid this practice. Literally just ask it to figure out how the product has changed from generation to generation, compare prices over time and between brands, and if needed see if the brand has changed ownership anytime recently.
Related to my thoughts on Private Equity buying out local businesses, particularly those with lengthy presence in a community.
Everything is owned by an increasingly small number of conglomerates who wear different skin suits to con suckers into buying from them, and not from those other guys, who are also them. It's starting to feel like a home-grown version of Chaebols, or Zaibatsu, and people are checking out.
On the one hand you could chalk this up to just inevitable outcome of Capitalism where all goodwill, consumer surplus, and 'brand loyalty' is converted into shareholder value whenever possible.
I think its not inevitable, but just as in nature, any excess calories will invite predators, scavengers, or parasites to 'restore equilibrium.' "Oh boy, people will pay a bit of premium on this particular brand to gain status/ensure functionality/avoid copycats. Let's see how much money we can pump them for before they balk."
I kind of disagree that "the entire concept of 'brand' has been eroding."
Its starting to appear like the brand is now the only factor that matters, when the individual consumer is not independently able to judge the quality of their products.
One of the more stark examples is the apparent widespread, entrenched preference for the iPhone over any competitor, even though the average smartphones are almost identical in capabilities these days. The higher end Samsungs are usually better than the iPhone in terms of cutting-edge tech. But Apple has a TON of lock-in and goodwill purchased during Steve Jobs' tenure, and current execs seem competent at maintaining that edge. (Yes, I know Apple software itself has some real advantages over Android).
It feels like MBA-types are very keen about recognizing a brand-name that has a positive reputation (even or perhaps ESPECIALLY if the brand is all they have, they don't own any manufacturing capacity), and then 'rug-pulling' the fans/aficianados who 'bought in' while it was on the rise to squeeze a burst of cash from them even while removing those factors/features they most loved about the product itself.
This is perhaps most annoying to me because I think 'brand loyalty' is mostly a good thing insofar as there's an 'implicit' contract that the company will keep its products of generally the same sort of quality and, one hopes, pricing as they've been, absent some external forces acting upon them and the customers are able to buy said products without having to do extra research to know what they're getting. And usually don't have to worry about the customer service because the company knows repeat business will come so keeping satisfaction high is prioritized.
When the company changes ownership and management, they usually go right about breaking that implicit contract (ADMITTED that there is no legally enforceable cause of action here!) but are happy to coast off customers' belief that little has changed, and certainly won't ever explain that they're cutting corners to save costs.
faceh
4d ago
I wonder if sometimes that's a natural result of forming a sufficiently complete/complex model of reality to basically explain almost everything you encounter, and yet realizing it is still incomplete, or unsatisfactory. You keep having to push back the explanation of 'where it all came from' to the big bang and then... what?
If there's not something else out there, and if that something doesn't transcend our understanding of physics, then how in the hell did we get the rules known as 'physics' in the first place?
And if you can't let that thought rest, it opens the door to many things being possible. Believing in ghosts isn't so 'odd' if you're already theorizing about alternate timelines and the existence of higher dimensions that could harbor intelligent life.
And of course, the belief in superintelligence being possible... well that creates an opening for all sorts of weird things that could happen to you.
faceh
7d ago
This + hormonal birth control has undoubtedly had massive downstream impacts. Probably on things other than pure TFR, too. Sometimes literally downstream, turning the frogs gay.
My strong opinion, but is 'loosely' held in the sense the data to support it is spotty, is that the 'phones' aren't the proximate cause. Its the algorithms.
Maybe this distinction doesn't matter because the smartphones are what enabled the algorithms.
The never-ending doomscroll, the optimization for engagement, the gamification and then the gamblification of all things, the centralization and enshittification of the web, the dominance of clickbait and later ragebait content, and I'll throw in the competitive ranking of every aspect of life, all of it ultimately driven by serving ads... I genuinely think that is primarily due to companies designing black-box algorithms that optimized for ads and pure attention-seeking, and away from prosocial, high agency, discreet good behavior.
That path may have been the most likely way things developed, but I think there were different outcomes possible if we had achieved certain political solutions or incentives had pulled in a different direction. If, SOMEHOW, 'serving ads at every possible turn' was NOT the primary way to finance the web's development. I guess that's also imagining the Google never becomes dominant and makes trillions of dollars from having the best advertising tech.
Eternal September would still be at thing, of course, even if we improve incentives the sanity waterline for the online world is already abysmally low. I just think that we COULD have hit an equilibrium where the online economy wasn't driven by attacking that sanity waterline, driving it lower and exploiting the differential. To somewhat overstate my point. Try to imagine, if you can, an app ecosystem that earned money from you only to the extent your life was actually improved by the use of the app, rather than ruthlessly trying to get you addicted to the point you spend money irrespective of the outcomes. Where social discord was considered an actual externality in the vein of pollution or toxic waste that should be internalized as much as possible.
Evidence that is kind of ambiguous would come from comparing the Western internet to the Chinese internet. I think it sort of supports my point because They cracked down on algorithmic feeds and have been more active on censoring types of content considered antisocial or corrosive. Notoriously their version of tiktok was more 'wholesome.'
I do not say here whether their censorship tactics are good or bad (I have my opinion), but I just offer it as a counterfactual to the West's situation. Maybe the actual outcomes aren't all that different in the end.
Ultimately it would be VERY dicey to separate out the variables here, especially with Covid throwing a wrench in everything.
This is also my short-term AI fear, BTW. Once they finally breach the norm against serving ads in your token stream, I expect bad things to follow in that space as well.
faceh
9d ago
That's partially my point. if you can massage the numbers then even rank incompetence isn't necessarily enough, it might require literal criminal charges.
I suspect Chicago has been that way for a longgggg time.
Yeah, I've pointed out that particular case before, where rank incompetence is enough to get you removed if you make your local community's lives noticeably worse.
Although Karen Bass is seemingly testing the limits of that premise.
Of course, Chesa got a high-paying, high prestige job RIGHT after getting booted, so the Dem machine takes care of their own too.
Not sure how credible.
Barely.
High estimate of 12% of the Bernie bros actually pulled for Trump is not nothing, but not quite a credible threat (to be clear, I don't think Fuentes' calls to vote Dem are a credible threat either).
If it were true, though, what’s the mechanism for tightening the leash? Are there a bunch of midlevel Dems who mellowed out to get a Senate seat?
I'd normally guess its simply funding and access to party resources to run a campaign, but in reality I'd bet they will straight deplatform or cancel them with kompromat.
Which is the approach they've applied with Platner, it seems. How many times has a Democrat pol who was gaining momentum gotten hamstrung by "allegations" and "leaks" when they run afoul of the national party line? I can think of a few offhand.
faceh
10d ago
This seems most correct. In the Trump era, there's less willingness to (publicly) swing over to the other side by Dems who are disappointed with their own politicians' performance. I'm sure many will do so privately yet I'd guess in lower numbers than years past.
So the risk for Dems is mostly that their candidate is so off-putting or blatantly unqualified that voters stay home. But (moderate) voters staying home would mean that the more extreme wing, by actually showing up, is driving more of the outcome. In safer blue districts, you'd expect this would mean the extreme candidates can win unless the GOP opponent is able to eke it out.
I dunno, I haven't seen a situation where some Dem candidate is so bad that it actually activates more of their base to come out and cast a vote against them in protest. I bet some local elections it has happened.
Although it is 'odd' that I notice alt-right figures like Nick Fuentes flogging the "Trump/Other righty politician is too friendly with Israel, we must vote Democrat to punish them."
You never, ever, EVERRRRR see a lefty extremist going full "Vote for Donald/The Republican candidates down the ticket to teach our side a lesson!"
I'd say this has resulted in somewhat of a 'dual strategy' where the Democrat establishment, which historically has a tight leash on its own national-level candidates, will let extreme candidates run without interference at lower levels, whilst yanking the chain and ensuring that such candidates can't reach the point where they are contesting less safe, more critical positions. I also think this strategy is starting to get away from them.
faceh
15d ago
Yep.
Its one of those situations where the constraints in place made it more 'special.' As a student, you expect to actually run into the players on campus. They were expected to spend a full 4 years there, and attend (easy) classes. They weren't millionaires (yet) but had a good shot at making it big in the NFL if they did well.
Ultimately, I'd be fine if the college football players make money from their likeness, but the team as a whole should still feel entangled with the college its tied to in ways that aren't easily severable, and that's just completely lost now. "Student-Athlete" just isn't really a thing.
I don't want to just randomly lay another problem at the Boomers' feet, but it sure looks like they're the ones who decide to break the system by both being the deep pockets that keep pumping up the athlete's values AND keep pulling out the guardrails on various behaviors and restrictions on the athlete's loyalties b/c they want to make their preferred team as competitive as possible.
Overall seems like another example of a 'runaway' system where everyone pursuing their immediate interests shifts the whole edifice elsewhere, at the detriment of the whole.
I have many thoughts on such things.
faceh
16d ago
Its kind of both.
Its very frustrating to grow into a world where sex is treated simultaneously as some uninteresting normal thing that everyone has and its no big deal, and yet also this critically important feature of social life around which one's personal status in the hierarchy revolves.
You go to college, and women can be very open about sleeping around, with no shame. But if you propose a tryst to one, out of the blue, with the implication that this woman is easy to get into bed, this is considered a massive, 'creepy' faux pas.
So its less "she wouldn't sleep with me" and more "why am I punished for seeking out sexual access when everyone else treats sexual access as a casual, unremarkable feature of college life?"
But there is absolutely the portion of this ecosystem, the semi-predatory male who specializes in getting naive women to open their legs that doesn't get remarked upon.
faceh
16d ago
Not the evidence I've seen.
Specifically, college educated womens' voting patterns (in the U.S.) vs. those of non-college educated women.
I would agree that their natural inclinations are towards the left, if anything.
faceh
16d ago
My suspicion is that the rise of 'useless' genders studies degrees and major overproduction of women with degrees took off in the 90's, hence why we didn't really get a swarm of SJWs until the 2010s.
Colleges were hives of leftist thought long before then, but that was counterbalanced by there being ample alternative routes for one's career before that point.
Also, look at the stats on women getting Masters and Doctorate degrees to see where the problem became SERIOUS.
I think 'intelligence' if defined in 'practical' terms is "the efficiency with which one can absorb and process the information in an environment, then utilize (or at least theorize how) the material in the local environment to achieve particular goals."
The more complex the goals one can achieve, and the more efficiently they can achieve them, the higher the intelligence.
The Von Neumann/Manhattan Project parallel I'm drawing makes this point. Given all the materials necessary to make a nuclear weapon, how quickly can a particular group of humans go from merely theorizing about the possibility to actually getting one built.
A group of humans that includes Von Neumann and other Physics PhDs, with the backing of the U.S. military, can get it done in, say, 5 years.
A similarly sized group of humans of utterly average intelligence (as measured by IQ)... probably never. Even WITH the backing of the U.S. military.
One Von Neumann and a bunch of average IQ humans... well I don't know.
A whole bunch of Von Neumans working together...
More options
Context Copy link