This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
It's been funny to watch NLP researchers (including corporate-affiliated ones) go through stages of grief, from their peak of jovial Marcusian pooh-poohing of the tech, to absolutely clowning themselves with childish misrepresentations, to jumping on the AI risk bandwagon, to what now seems like anhedonia. No doubt Altman and co.'s deft market capture strategy and development velocity are crucial factors here. Altman is known to be… well, I'll let Paul speak of this.
But I suspect this has more to do with dismal unforced errors of other groups. Technically, many ought to have been more than strong enough to pose a challenge and, indeed, all of this revolution is mostly of their making. Their failure to capitalize on it reminds me of those Mesoamerican toy wheels and planes, and of the Chinese firework-rockets and useless intercontinental fleets. It takes a special kind of mind to appreciate the real-world power of a concept; but that's not the exact same kind of mind that excels at coming up with concepts, and not necessarily even the one that's best at implementing them.
I'd even say that the fact that OpenAI is now making safety noises and withholds general facts like the parameter count is telling: this is how little technical moat they have.
What they might realistically have is the cultural moat: theirs is the culture laser-focused at transformative, dangerous AGI through deep learning, from their idealistic beginnings to prevent the zero-sum race, to their current leading position in it. They enforce their culture through a charter which their employees, I've heard, learn by heart. Dynomight has argued recently that what you need to begin making explosive progress is a demo; they've had the demo inside their heads all along.
This cannot be said for others.
The French gang at Meta is represented by the archetypal Blue Tribe intellectual LeCun, and he… he too is dismissive of the fruit of his own work. Like Chollet, who focuses on «interpolation» and «compression» in AI as opposed to genuine comprehension, he advocates for clunky factored bioinspired AI, he speaks of the shortcomings of transformers and their non-viability for AGI – too pig-headed to sacrifice a minor academic preconception. They're too rational by half, they lack the requisite craziness to jump over that trough in the fitness landscape, and believe in science fictions, sell the half-cooked snake oil to the end user, and fake it until they actually make it – a typical French engineer problem. They've published Toolformer, but it's ChatGPT Plugins which blow people's minds – being essentially the same tech.
The Googlers, on the other hand, are handicapped by their management. Again, it cannot be overstated how much of Google's research (actual Google Brain research and Deepmind both) has laid the groundwork for OpenAI's LLM product dominance; and with barely any reciprocal flow. GPT-4, too, is almost certainly built on Google's papers. They have optimized inference, and training objective, and every other piece needed to turn PaLM or Chinchilla into a full-fledged GPT competitor, and they even have their own hardware tailored for their tasks, and I think they've wasted much, much more compute. Yet they have not productivized it.
I strongly suspect we should blame the Gervais Principle, and the myopic board of directors that gets impressed with superficial Powerpoint bullshit. The worst offenders per capita may be Indians: while their engineers can be exceptional (heck, see the first author of the original Transformers paper), the upper crust are ruthless careerists, willing to gut moonshots to please the board with rising KPIs and good publicity when they get into management, or obsessively funneling resources into their own vanity projects. Many corporations have already suffered this catastrophic effect, exactly when they tried to reinvent themselves in response to novel pressures – both Intel and AMD, even Microsoft. IBM isn't doing too hot either, is it? Was Twitter prospering under Agrawal?
But of course it's not specific to Indians. I've heard that the guy behind the infamous LaMDA and now Bard, which is so clearly inferior even to ChatGPT 3.5 version, Zoubin Ghahramani, has been very skeptical of deep learning and prefers «elegant» things like Gaussian Processes – things you can publish on and inflate your H-index, one could uncharitably state. Also he's a cofounder of Geometric Intelligence (yes, Gary Marcus strikes again).
Social technology doesn't always trump engineered technology, but by God can it shoot it in the foot.
Love this writeup. To be fair to Zoubin though, he was Geoff Hinton's postdoc in 1995, and worked on deep learning in way, way before it was cool. It's just that deep learning didn't really do anything on those tiny computers. You might say that this makes it all the more unforgivable that he slept on deep learning for as long as he did in the 2010s. But Gaussian processes are infinitely-wide neural networks! And the main sales pitch of Bayesian nonparametrics was that it was the only approach that could scale to arbitrarily large and complex datasets! Pitman-Yor processes and the sequence memoizer were also ultra-scalable, arbitarily-complex, generative unsupervised language models that came out of those approaches. But scale isn't all you need, you also need depth / abstraction. But before transformers, depth seemed to lead to only limited forms of abstraction, and doing something more like a search over programs seemed more promising.
More options
Context Copy link
It seems a variation of the Innovator's Dilemma. Individual researchers and engineers at OpenAI aren't too different from those at FB or Google: most of them worked at Brain or DM or Meta or had LeCun as their advisor or helped rig the sails on Jeff Dean's treasure fleet. But Jeff Dean was never going to burn his boats and wouldn't have been able to even if he were inclined to; why should he, when the tributes from Keras and Jax were so small and had no influence over any faction of the eunuchs and bureaucrats who ruled the court in the imperial center?
Sam Altman had no constituency except capital, and so he could make his bet that seems, in retrospect, obvious.
More options
Context Copy link
There is much truth to it. Indian managers (on average), while brilliant, are held back by their culture of deference to the experienced and cultural incentives to not rock the ship. 200 years of being Bureaucrats to the British, and they remain Bureaucrats in even independence.
No one can meet quarterly goals quite like a Bureaucrat. No brings a golden goose down to a halt quite like a Bureaucrat. The "Hindu rate of growth", insulting as it was, pointed fingers squarely at the Bureaucracy for it's relative stability and sorry growth.
Microsoft is the counter example. the work that Satya has done at Microsoft consistently impressed people through the last decade. Indian careerists make terrible business leaders. But Indian businessmen are an entirely different ballgame. Sadly, both groups don't fix.
Can you elaborate on this? As the descendant of intermarried Scots, I'm curious what you mean.
More options
Context Copy link
Razib has written a lot about this. Both groups are some of the most endogamous groups dating back (around 1500 yrs) further than even Ashkenazi jewish endogamy.
You can't forget the trader class (Marwadis, Sindhis) if we are talking about comparing them to people whose caricatures are money-lenders with exaggerated features. The Parsis are also incredibly similar. Rich, endogamous, genocided and now flourishing in their new refugee liberal home. The Parsis need to learn from the Orthodox Jews and start having unprotected sex. They're going extinct.
While the I would love to take south-asian over-representation on this forum as an indicator of high verbal-IQ, I think there is another factor at play here : Colonialism. Most Indians on here are 1st generation immigrants. A lot of the top comedians are either 1st gen immigrants (Kumail, Hasan) or grew up away from 'white America' (Nimesh in NJ).
2nd gen immigrants (Indians and east-asians) are desperate to integrate into normie white culture. They will never end up in a place as transgressive as this. The 1st gen is best suited to hang out here, but the 1st gen east-asians simply do not speak great English. I do believe east-asian conformity doesn't lend itself well to forums like ours, but to me, the other 2 factors play a bigger role in their absence.
Ah, while my parents are also in the UK I wasn't born here, I came with them to this country during my early teenage years and I think my basic mental view of the world had part formed before I stepped foot on Western shores. A significant part of my schooling took place outside the UK. So I'm not really second gen (but equally not really first gen either).
If you were to ask me what I see myself as I would without a doubt reply that I am Hindustani and not English.
No matter how I dress or talk, or the way I interact with other westerners in the end phir bhi dil hai Hindustani. And I am and will always be proud of it.
Clearly. Your dislike, and secular arguments against, western modernity sound about as genuine as qutb's, or bin laden's complaints of american imperialism.
Interestingly I have very negative opinions of both those people. I dislike violence on aesthetic grounds and it is almost never necessary to deploy it to achieve your goals. There are often much kinder ways to get what you are after. See how the British used violence to conquer my country, but now we are going to conquer them with love (by having more kids and letting mathematics do its job). No violence or threats needed, only love.
Sure you don't like them. Islam spread by force of arms, and as soon as war took more than riding a camel/horse/elephant into battle, they sucked at it. Changing strategies might be a good idea if they were capable of it. Left to their own devices they degenerate into isis or at best pakistan. Anyway that's not the point. No matter how many West-critical studies you quote or disgust you express at this or that western quirk, those things are not your true objection. You can't reason someone out of a position they didn't reason themselves into.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Satya is a clear counterexample, yes (or as you say, he comes from a different career track? Both him and Pichai started out as hard engineers and pivoted into management, both come from higher-tier Brahmin lineages… @2rafa, do you know anything? I've only heard that Pichai has great… mediating skills).
I am thinking of some other high-ranking manager who has departed recently; but his name can't escape the tip of my tongue. Maybe some Ramakrishnan.
In general I am curious as to the reason for people like Pichai's meteoric careers. It can't be that easy to become CEO of a trillion-class corporation with major strategic value, the competition must be immense. Why have Brin and Page, with Schmidt's input, decided to leave him in charge?
As the best people are wont to do. Very little beats strong technical ability combined with good people skills.
More options
Context Copy link
Google's board was heavily influenced by Bill Campbell, a Svengali-like figure in Silicon Valley. He like the cut of Sundar's jib and chose him as the bright young thing that should be promoted. Most of Google's board was in awe of Campbell, so gave the nod to Sundar when it came time to put a PM in charge of Chrome, replace Andy at Android, replace Alan as boss of all engineering, and then replace Larry as CEO. It is difficult to capture quite how much influence Campbell had on Google's promotion decisions. Even after his death, Google's board would ask "What would Bill say?" Why Campbell liked Sundar is another question entirely. Sundar is not technical at all - his undergraduate and masters is in materials science, which has nothing to do with IT (well, outside of chips). Bill liked non-technical, slightly unpolished people. It may be that Sundar was the one he met that day.
More options
Context Copy link
I don't know your mother, who may well speak softly and authoritatively, but I don't think Sundar is like that at all. He always managed up, and once he achieved positions of power, completely ignored his reports. No-one claims that Android was more successful under Sundar than it was under Google. Then, when Sundar ran all of engineering, I don't think anyone can point to something achieved during that time, other than the huge success of AI research. It is hard to give Sundar credit for that, since he completely mismanaged bringing that work to product, and let Google, who did most of the research, be eclipsed by OpenAI. Since he became CEO in 2015, it is hard to point to a successful new Google endeavor or product. This contrasts with Satya, who meets with perhaps too many people. If Sundar is known for anything, it is being indecisive and failing to make decisions. On the other hand, not making any decisions turned out quite well for Google for at least the first five years of his tenure. We will see if Sundar's unwillingness to act resolutely is Google's undoing.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link