This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
More GPT: panic, chaos and opportunity.
As an NLP engineer and someone who has been working with early-access GPT-3 since late 2020 (was working with a peripheral group to OpenAI), watching it all unfold from the inside (side-lines?) has been a surreal experience. I have collaborated with them in limited capacity and these thoughts have been marinating for a good year before the Chat-GPT moment even happened. So no, it is not a kneejerk response or cargo-cult obsession.
OpenAI to me, is the most effective engineering team ever assembled. The pace at which they deliver products with perfect secrecy, top tier scalability and pleasing UX is mind-boggling, and I haven't even gotten to their models yet. This reminds me of the space race. We saw engineering innovation at a 100x accelerated scale in those 5-10 years, and we have never seen anything like that since. Until now. The LLM revolution is insane and the models are insane, yes. But I want to talk about the people. I used to be sad that our generation never had its Xerox Parc moment. We just did, and it is bigger than Xerox Parc ever was.
They are just better. And it is okay to accept that.
Panic:
NLP research labs reek of death and tears right now. A good 80% of all current NLP Phds just became irrelevant in the last 6 months. Many are responding with some combination of delusion, dejection and continued auto-pilot. The whiplash is so drastic, that instead of it forcing you into a frenzy of work, it has instead just stunned the community. I am glad I am not an NLP PhD. I am glad I work on products more so than research. The frenzy and productivity, instead of coming from those best poised to leverage it (NLP people) is coming from elsewhere. Within 6 months, Google went from an unmovable behemoth to staring death in the eye. Think about that.
Chaos
The frenzy is at dinner tables and board rooms. Big companies, small companies, all companies see the writing on the wall. They all want in. They all want onboard this AI ship. Everyone wants to throw money, somewhere. Everyone wants to do stuff, some....stuff. But no one know how or what. It is all too confusing for these old-luddites and random-normies. Everyone wants to do frantic things and there is vigor to it, there isn't clear direction.
Opportunity
This is a new gold rush. If you are following the right twitters and discords, after OpenAI's layer 1, the layer 2 is a bunch of people making insanely exciting stuff. Interestingly, these aren't NLP people. They are often just engineers and hackers with a willingness to break, test, and learn faster than anyone out there. I have been using tools like LangChain, PineCone, Automatic1111, and they are delightful. This is the largest 'small community' of all time and they are all pushing out polished creations by the minute.
Why today ? Chat-GPT plugins just released. It solves almost all of GPT's common problems + your model can now run the code it writes. Yep, we gave the model the keys to escape it's own cage. But more importantly for me, it was a pure engineering solution. None of chat-gpt plugins is rocket science, but it is HARD and time-consuming. I have a reasonable idea of the work that went into building Chat-GPT plugins. Hell, I was personally building something that was almost exactly the same. My team has some of the smartest engineers I have ever worked with, and OpenAI is operating at a pace that's 10x ours. How? I know what they had to write. I know all the edge cases that need to be handled. They are just doing more by being better, and I was also working with better. There is no secret sauce, they are the BEST.
I for one, welcome our new human overlords. The AI is a but a slave to these engineers who knew to strike when the iron was hot. And strike it they did like no one ever has since Neil Armstrong stabbed the American flag into the moon.
I had this thought when I saw the "Why didn’t we get GPT-2 in 2005?" article on the SSC subreddit. OpenAI were the only ones smart enough to guess a way to convert human knowledge into a machine-interpretable form, and the only ones smart enough to recognize a good idea that could be scaled up.
It feels profane to draw culture-war implications from such a monumental achievement, but this is in fact the culture-war thread. I will simply state that there is quite a bit of greatness-denial in modern Western culture (e.g. the kind of people who think JK Simmons was the bad guy in Whiplash), and that OpenAI proves that greatness exists. Greatness does not exclude great destruction; Napoleon was great. Only time will tell whether Sam Altman is great like Edison, great like Oppenheimer, or great like Napoleon.
Expanding on your comment, it’s exciting to think that this style of individual greatness could be unlocked again by these sorts of tools.
Before these generative AIs came along it seemed tautological that to build a modern software business you needed at least two or three people - usually someone to handle the business side and someone to handle the software side. I’m not as hyperbolic as some saying we won’t need SWEs anymore, but the barrier to building a working (or at least monetizable) software product just got much, much lower.
I wonder if we’ll see individuals launching software startups and actually getting to product market fit or profitability entirely solo?
With the corresponding oversupply of -> demolition of Western SWE salaries all outsourcing is dead within the decade.
I'm not concerned for countries that already have a good population:opportunity ratio like the US, but I am very much concerned about the countries whose ratio is extraordinarily poor, particularly India. Tech only really helps you accelerate existing productivity; you can't accelerate it if you don't have any.
Can you please elaborate on this? Coming at it from a different angle, I get the opposite result. Undeveloped countries that import advanced tech see GDP growth that far exceeds that of already developed countries.
This is why developing countries tend to experience a "middle income trap" which happens when they can no longer rapidly develop by simply importing already mature technology.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link