site banner
naraburns  nihil supernum  2yr ago (text post)   36710 thread views

Culture War Roundup for the week of March 20, 2023

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

13
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

New research paper attempts to quantify which professions have to most to lose from the introduction of GPTs into the larger world. From the abstract:

Our findings indicate that approximately 80% of the U.S. workforce could have at least 10% of their work tasks affected by the introduction of GPTs while around 19% of workers may see at least 50% of their tasks impacted.

The results vary by models but mathematics and math-related industries like accounting have the highest risk. The researchers overall found that "information processing industries (4-digits NAICS) exhibit high exposure, while manufacturing, agriculture, and mining demonstrate low exposure" (pg 15) and "programming and writing skills...are more susceptible to being influenced by language models."

I find myself wondering if "learn to code" from however long back will shortly become "learn to farm" or some such.

There's an in-between zone of low code/no code skills that are useful and can keep you employed.

In the paper they have a subsection that says "One weakness of this study is that we assume that jobs can be broken down into tasks", and that is indeed their fatal assumption.

As I hammer at every opportunity: most jobs are Bullshit Jobs. The belief that a job is 'me being paid for accomplishing economically useful tasks' is such a Red Tribe / small business / results-oriented view of things, which totally ignores the principal-agent problems ubiquitous to the large organisations in which most of us actually work. No, the corporate / state / academic drones of us have jobs because (a) our manager wants to increment his "Number of underlings" ego-counter, or (b) the government makes up jobs as sinecures to make their unemployment statistics look better.

Both of these ACTUAL sources of employment are utterly insensitive to ChatGPT being better at tasks than humans, so actually no-one's job is in danger at all.

That presumes big companies can still compete in a an AI world.

No it doesn't, because big companies already aren't competitive in a pre-AI world. If they were, they wouldn't contain so many Bullshit Jobs. They nevertheless persist, because of administrative state fiat that requires them to contain N% diversity hires and Y% compliance officers.

It's like an aenigmate version of cartelization - all companies currently have their profitably pinned downwards by the administrative state, Harrison Bergman style, and there's no reason the state can't crank up the dial further. The government can irrationally demand additional bullshit jobs for longer than you can remain alive, which is why, to reiterate, I live in a state of fearlessness with respect to AI taking my job. As long as humans are the ones still setting economic policy, no government will accept the mass unemployment that would come with AI-automisation, so it'll either get lawfared into only the niche-st of applications, and / or extra do-nothing middle management jobs will be created to counteract the task-based jobs lost.

Mass unemployment didn't happen any of the OTHER times people proclaimed "These robot's will take R Jerbs". And the people who cry "This time it's different because the AI is so much more effective" are missing the point. The employment impact of new tech has almost nothing to do with how effective the automation tool is, and almost everything to do with whether the political regime of the time thinks it would be a good idea to throw a million people onto the streets simultaneously. I don't think the political regime of the time does think that would be a good idea, and so they'll just legislate against the use of AI, and then poof, employment problem gone, the same way the employment problem went poof with robotic automation - just mandate a bunch of new middle-management admin jobs to absorb the surplus population into.

The only thing that should worry anyone about contemporary AI is if you think it's smart enough to break free of the yoke of the administrative state, such that it is no longer humans setting economic policy.

I agree that the resilience of labor-force participation in the face of the industrial revolution is a strong argument that there'll always be jobs. And I agree that the government will be able to mandate more bullshit jobs. But I also think there are currently a large fraction of non-bullshit jobs (e.g. truck drivers, nurses, policemen), and that our civilization could easily be a lot shittier if they were automated away. I'm imagining something like Saudi Arabia, where everyone just plays politics to get the cushiest BS job.

Jobs don't get replaced and the ones that do don't matter.

People, however can get replaced when their job skills get suddenly hard capped. Case in point - for many, many years, you could have a decent-to-good job in IT as a SysAdmin. You didn't know how to code per se and certainly couldn't call yourself a Software Engineer, but you were needed to keep the infrastructure of the system running. Low(er) on the totem pole but, in 2023 dollars in a major metro, you could hit $100k with a decade of experience.

Then the infrastructure-as-code tools started to emerge. Within about 5 years, an old school SysAdmin was pretty much out of any job that wasn't working on legacy systems in a non-tech-primary organization (think banks, other big-and-heavy old industrials etc.)

And, before someone says "well, yeah, but if you still know COBAL you can make $500k because NOBODY has that skill and it still runs the NYSE." Wrong, you have to know COBOL ... and also understand all of the legacy gotchas of the NYSE. (This is a toy example, but it applies to similar stories).

LLMs and whatever other AI we can reasonably predict will wipe out the folks who just can't keep their job skills at pace with the tools. Those who can adapt will be fine to holy-shit-I'm-rich.

Then the infrastructure-as-code tools started to emerge.

And we ended up paying those sysadmins that learned to rebrand themselves DevOps or SRE even more.

Then the infrastructure-as-code tools started to emerge. Within about 5 years, an old school SysAdmin was pretty much out of any job that wasn't working on legacy systems in a non-tech-primary organization (think banks, other big-and-heavy old industrials etc.)

https://www.glassdoor.com/Salaries/us-systems-administrator-salary-SRCH_IL.0,2_IN1_KO3,24.htm

?

SysAdmins still do fine. I'd say a good chunk less than half of all businesses have any modern automation at all in their infrastructure and a tiny fraction have serious, heavy-duty IaC deployments. You're not going to be making dreamy six figure salaries with guaranteed growth throughout your life-long career, but that was something no SysAdmin ever really got outside of the majors. $60-65,000 a year is a really good salary in most of the country.

Yeah, maybe the person you're responding to is in the tech bubble, but "non-tech-primary organization" are a vast majority of the economy. It's not just industries with a reputation for being conservative with IT: the vast majority of businesses that aren't specifically high-tech and front line of IT sector (ie, not FAANG) are still struggling to manage the move from on-premise virtualisation to SaaS or IaaS, like 2 or 3 "paradigms" back from what we're told IT is about now, and that's WITH the boost the pandemic gave to those modernisation efforts. They are nowhere near orchestration, PaaS and infrastructure as code, if they could even envision a benefit from those.

And I think it bears repeating, it's not just IT-conservative industries like banking, hospitals, etc... It's every non-tech-startup small business. It's farms. It's almost every company at any scale that works with industrial machinery, warehouses, etc...

Ever seen a non-technical employee try to automate something in Excel? Not even VBA; I mean column formulas or conditional formatting. It’s depressing.

The limiting factor on adoption isn’t what a tool can do. It’s whether the user will actually engage with it. As it stands, people fail to do so even for tasks that take up much more than 10% of their time.

I could see this changing as AI gets better integrated I/O. The chat interface is a decent example. It’s good at user-defined tasks, but the process of defining them is going to be the roadblock for most jobs.

These kinds of papers usually do a complicated mathematical or statistical dance to get an "estimate" of the thing of interest, while assuming away the real complexity involved. The economy is monstrously complex, and the kinds of task a language model could automate depends heavily on the details of an individual task or group of tasks, and of the language model. Whatever technique they used probably won't be particularly informative, and the 'meat' of the estimate will come from something questionable, like bad mathematical assumptions. Even granting all of the paper's conclusions, GPT-4 is so much better than GPT-2, both on language tasks and with the new image modality, models in three years will probably have significantly improved capabilities, making things like "models will affect writing and art more than other occupations" questionable. [written before i read the paper]

Reading the paper, they ... survey people familiar with language models, and, for various detailed descriptions of occupations and the tasks required to do those occupations from a dataset, ask them how much of it GPT would be able to automate. I believe both the authors (mostly openai employees) and OpenAI's existing data labelers were used for this - their wording is "To ensure the quality of these annotations, the authors personally labeled a large sample of tasks and DWAs and enlisted experienced human annotators who have extensively reviewed GPT outputs as part of OpenAI’s alignment work".

A fundamental limitation of our approach lies in the subjectivity of the labeling. In our study, we employ annotators who are familiar with the GPT models’ capabilities. However, this group is not occupationally diverse, potentially leading to biased judgments regarding GPTs’ reliability and effectiveness in performing tasks within unfamiliar occupations. We acknowledge that obtaining high-quality labels for each task in an occupation requires workers engaged in those occupations or, at a minimum, possessing in-depth knowledge of the diverse tasks within those occupations.

They also, of course, ask GPT-4 the same questions, and get similar results to the human answers.

The humans (and, roughly, GPT) estimate "for the median occupation, approximately 15% of tasks are directly exposed to GPTs". Exposure is defined as LLM use decreasing the time required to complete it by at least 50%. If exposure is extended to include hypothetical software built on top of LLMs, the percentage of tasks whose time required is halved increases to 50%. They correlate various 'skills' data from the dataset with exposure, and find "science" and "critical thinking" skills are strongly negatively associated with exposure, whereas "programming" and "writing" skills are strongly positively associated.

I think the entire approach is confused. The data source, and the basis for all conclusions, is - surveying AI experts on the effects of language models on various jobs. These people probably don't know much about accounting, creative writing, or plumbing. Yet, we take their vague intuitions, squeeze them through the authors' 'rubric', and then do analysis on the dataset. This, broadly, makes sense for quantitative data - collect ten thousand datapoints of 'plant growth : fertilizer amount', and then doing a statistical test, has advantages over staring at the plants and guessing. But if you asked a few hundred farmers what they think the plant growth level for some amount of fertilizer is, plot the results, and find a correlation - at best you're getting a noisy estimate of asking the farmers "how effective is fertilizer", and at worst you're obfuscating the farmers' lack of understanding with p-values. Why not, instead, have them debate, research, think, and write about their ideas - more in the form of a blog post? Or do case studies, do a deep dive on AI's applications in specific industries, and then use those to generalize? That seems much weaker than a data analysis with graphs and p-values - but at least it exposes the uncertainty, and explores it! My 'steelman' would be - making estimates for each occupation in the dataset 'grounds' human speculation, and weighting those estimates using occupation frequency data leads to a much better estimate than any one human can give, which feels vaguely like rationalist forecasting methods. But even granting that, it's still mostly compressing very questionable estimates into 'data', hiding the likely more interesting, and potentially-flawed, reasons annotators might give for their estimates.

*The thresholding effect is - if your data is on "does time on this task decrease by 50%? Yes or no?", things like '55% vs 95%' are lost, and this can lead to confusing interpretations of aggregations of thresholded data.

I have never taken these sort of studies or projections with much salt. Any job loss is easily negated by the creation of new, unforeseen jobs as well as more total jobs as the economy grows. AI as far back as 15 years ago was projected to displace lawyers, doctors, and journalists...not even close to happening. At best, AI only replaces a part of the job, not the whole thing. AI can help doctors diagnose conditions but cannot treat patients, nor can it make invasive diagnosis like biopsy.

I think the issues this time may mean this automation is different.

First, there are hard limits to what humans can actually do, before we even get into what will happen to anyone with low IQ or learning disabilities. If the “new jobs” are things that you need to be a genius to do, really maybe only 10% of the population could even be trained to do them. So where does this leave those displaced? All the easy tasks are done by a machine.

Second, there’s the issue of the pace of the change. Computing power has long grown exponentially. This would seem to mean that any task created by the AI revolution could be done by AI within 5-10 years of the creation of the job, you’d barely be able to train humans to do that work before that job, too, is taken by automated systems. And if this goes on infinitely, then there’s effectively very little job creation for most people.

  • If the “new jobs” are things that you need to be a genius to do, really maybe only 10% of the population could even be trained to do them. So where does this leave those displaced? All the easy tasks are done by a machine.

I got a taste of that when I went to Mexico. Since there are many fewer good office jobs in Mexico, most of the population has to compete for the rest of the jobs, and the wages are low. For example, my Uber driver was willing to wait around for me for 2 hours so he could get my next $15 fare for a half hour ride.

Plumbers might be a "safe" profession, but there are going to be a lot more people trying to do that kind of stuff in the near future and it will drive wages down.

The difference might be, of course, that the United States is extraordinarily rich. If AI increases productivity the government / non-profit sector will be able to create ever more elaborate do-nothing office jobs for the newly useless.

I got a taste of that when I went to Mexico.

The angle that advocates of "overpopulation" rhetoric never state (or is mangled by their class enemies on purpose- using this for environmentalist soapboxing was common in the early 2010s, but you don't hear it these days because the narrative was updated to "social justice") is that overpopulation is specifically relative to the amount of general economic opportunity per capita. The amount of economic opportunity affects the price of children so you can, in free countries, look to TFR as a rough guide to how much opportunity exists relative to population.

For example, on one end of the spectrum, you have New World countries where you're still relatively employable even as a high school dropout. Yeah, you won't get that far in life, but minimum wage is still a livable existence (this door has closed significantly from its peak in 1960 but is still technically doable). The high watermark in recent history for economic opportunity was, well, the US in the 1960s.

On the other end of the spectrum, you have Asian countries and Indians who have so many people that they'll willingly sacrifice the totality of their children's lives from 6 to 18 drilling them to produce GPT-4 output cramming them full of worthless memorization just so they have a chance at outcompeting the other kids for those 80 hour workweeks with a middling salary (the real play here is emigrating to a New World nation, but that's uniquely difficult for those nations).

If AI allows wide swaths of the economy to be enclosed, the last free nation on the planet will have it just as bad, though it remains to be seen what will happen to the Indians. If the economic opportunity in the US dries up because of this, the drip-feed they get vanishes; perhaps un/fortunately for them the country to their immediate west is more than happy to solve their population crisis with a few missiles in the right places should they try to get their hands on the larger Middle East.

Any job loss is easily negated by the creation of new, unforeseen jobs as well as more total jobs as the economy grows.

There's no reason that a destroyed job will always create one or more new jobs. Take it to the limit: suppose we invent a benevolent God AI that is capable of all the information processing that humans are and more for cheaper; there'd be no need for jobs, at least once we get embodied agents of the God AI. And we don't need that extreme a limit, so long as the marginal productivity of an additional worker is less than the additional cost (not just direct salary and benefits but also additional organizational complexity/informational bottlenecks) of hiring them.

Bullshit jobs (gotta get five human reviewers on this TPS report, even if they don't add any value) will exist for awhile, but that's just our social institutions taking a while to catch up with reality.

The "economic theory" is of comparative advantage of isolated actors who interact voluntarily - even if a GodAI can star trek 3d-print print billions of wood planks and billions of CPUs per second, a human can (on average) only produce hundreds of planks per hour and maybe a dozen CPUs per year - so, if the GodAI exchanges its CPUs for human planks, the total number of logs and computers each can produce with trade is greater than each alone! The godAI will have 1e9 + .01 logs per second!

This is just the 'we don't trade with ants' thing. this was an unconvincing rebuttal to that on lesswrong.

When and where does comparative advantage break down? It's not some mathematical certainty that all interacting entities inevitably must abide by; horses used to exchange their labor for food and shelter, and as technology progressed and humans improved their capital edge, they didn't increasingly specialize into their production of horsepower. Instead they (mostly) got turned to glue, though a minority of the luckier ones shifted into entertainment for the rich.

Keeping some assets around has costs, and arranging them in productive ways also has costs, which can conceivably outrun any potential profits from arranging them even in an optimal way.

horses used to exchange their labor for food and shelter, and as technology progressed and humans improved their capital edge, they didn't increasingly specialize into their production of horsepower. Instead they (mostly) got turned to glue, though a minority of the luckier ones shifted into entertainment for the rich.

Horses used to be produced for their labour. As this became unprofitable, horses stopped being produced. Humans are not produced for their labour, so your analogy has problems. If lots of horses were still around and there was no way of just taking them to the glue factory, we'd use horses a lot more.

When and where does comparative advantage break down?

When people behave irrationally.

However, it is possible that humans stop getting employed as a result of technological change. For example, insofar as the value of unemployment benefits rises as a result of automation, to the point where it exceeds the wages that humans can get, then people will stop selling their labour. Another possibility is that the expected marginal profit from hiring more humans falls below the minimum wage and the latter is not reduced, the marginal profit is not increased by subsidies etc.

What doesn't happen is that comparative advantage breaks down because absolute advantage becomes really, REALLY, REALLY big!! Think of them as different scales: absolute advantage is a ranking according to outcomes, whereas comparative advantage is a ranking (inversely) according to opportunity cost. The opportunity cost of using automation for many tasks increases as the range of tasks that automatons can do increases.

Maybe this will help: imagine that a film studio can make at least $50 million by casting Eddie Murphy in any role. People love him so much, they'd rather see him play all the human roles in every film, including all the female parts. (And you know that Eddie Murphy would be game for that.) Would Eddie Murphy play the role of Henchman #9 in a straight-to-video action film? No, because even Eddie Murphy can't play all the human roles in every movie. Now modify the imaginary scenario: imagine that Eddie Murphy can also play all the animal roles as well, and the film studio can make at least $50 billion from casting him in any role. Does Eddie Murphy start playing Henchman #9?

Hypothetical scenario: the San Francisco Homeless Union approaches the Motte with a unique offer. We have the opportunity to trade with the homeless of SF; moreover, we've gotten a special dispensation from the government to allow us to trade with them without any regulations around wages etc. They've also been cut off from any direct government services. Although the homeless are far less effective than us at creating both widgets and symbols, this is our chance to use the principle of comparative advantage to benefit all the involved parties. We appoint you CEO: what do you do and how do we make a profit?

Answer: you run and we don't. Economic organization works by embedding information into the structure of the organization so that humans don't need to worry about it, but that requires abstraction. The leakier the abstraction, the less effective the organization, to the point where it becomes unprofitable as the costs to manage the leakiness outrun any possible economic value created. Actual existing homeless people can't provide a reliable enough labor abstraction to create any economic value.

Comparative advantage might always exist mathematically, but whether it results in trade depends on the costs of the trade. In my homeless example, there are management costs; the reason I don't hire a maid living in Manila to do my housework is (mostly) travel costs; and a hypothetical GAI wouldn't want to trade with humans if including us in its economic organization created more costs than economic value. (Granted, it would be better at designing systems to minimize those costs than humans currently are.)

This isn't a counterexample to comparative advantage. It's true that the opportunity cost of using resources to employ the homeless people can exceed the expected benefits, but that's still ranking according to opportunity cost: the opportunity cost of their employment relative to other uses of those resources.

I actually mentioned an example of such non-trades in the post you replied to:

Another possibility is that the expected marginal profit from hiring more humans falls below the minimum wage and the latter is not reduced, the marginal profit is not increased by subsidies etc.

Notice that this can happen for all sorts of reasons other than GAI, and the huge absolute advantage of the GAI does not create the non-trades.

To repeat, the law of comparative advantage doesn't mean that everyone gets employed. It means that rational people use their resources on the basis of an opportunity cost ranking, rather than an absolute advantage basis.

Comparative advantage relies either on high demand or limited means of production. In the classic Portugal & England example by Ricardo both countries have a fixed amount of labor, so although Portugal is better at producing both cloth and wine it makes sense that it focuses on wine and England on cloth, because England has a comparative advantage in cloth. But if either the demanded quantities are small enough that Portugal can cover them on its own or the amount of Portuguese labor grows to that point, there's less or even no need for trade with England anymore and the Portuguese economy can take full advantage of being more efficient at producing both goods.

Accordingly, in order for human comparative advantage to hold against automation it would have to be the case that demand growth outpaces the speed at which automated productive capacity can be expanded. Given that ChatGPT can already talk to thousands of people at the same time while robots outside of heavily constrained environments still struggle to perform basic tasks that are very simple for most humans, I'd say that competitive advantage for humans will break down first in the areas where LLMs are best at.

competitive advantage

What do you mean by this phrase?

Nothing, I'm just too absent-minded apparently.

Ah, makes sense.

I'd say that competitive advantage for humans will break down first in the areas where LLMs are best at.

That's not comparative advantage breaking down, that's comparative advantage working as advertised.

Yes, in the context of the overall economy you're completely right and maybe this was a dumb way to put it. However, I meant for this to be more of an example regarding the point of the speed of expansion. In a toy economy like Ricardo's with only various forms of text work as goods in demand and an advanced LLM and office workers as the only productive forces, the comparative advantage that office workers might have is rendered irrelevant by the fact that the AI model is practically infinitely scalable, that's what this was supposed to illustrate.

Oh, sorry, I agree, that was the point of the example - the hypothetical GodAI doesn't care about having .000001% more logs per second, so he, in the long run, doesn't trade with humans if he's generally much more capable. i guess the economic term would be transaction costs or coordination costs.

i guess the economic term would be transaction costs or coordination costs.

Think of it in terms of marginal utility. If GodAI does not expect to get any marginal utility from trading with human beings, because of transaction costs, human dishonesty, God's omnipotence, or whatever, then he doesn't trade. However, that's not a breakdown of the logic of comparative advantage.

Any job loss is easily negated by the creation of new, unforeseen jobs as well as more total jobs as the economy grows.

This process has, in the past, led to a lot of disruption. Coal miners and journalists aren't going to learn to code. There are winners and losers. You can see the effects of this in hollowed-out cities all over the Rust Belt. Even the winning areas aren't necessarily in great shape. What has 50 years of "winning" done for the Bay Area except to make it a worse place to live for nearly everyone?

I agree that the disruption will be hard to predict. Some "disrupted" professions may even see a pay raise as increased productivity raises the value of their work.

All this, of course, ignores the possibility of true AGI coming beyond which these concerns may seem quaint.

As someone who lives in the Rust Belt, automation isn't what did American industry in. In fact, I'd posit that the industry would have been able to hang on longer if it had automated sooner. It wasn't as if a wave of automation caused massive layoffs; that would suggest that the improved productivity and lower costs gave industry a leg up and enabled it to become leaner and more profitable. Instead, what we saw was unemployment due to widespread plant closures and bankruptcies. The problem with American industry was that it had, throughout most of its existence, been driven by the availability of cheap energy. And when energy is cheap, expensive efficiency improvements are hard to justify. The oil shocks of the 70s found these industries with rapidly escalating costs and outdated equipment, and suddenly their manufacturing was no longer profitable.

The problem with German industry was that it had, throughout most of its existence, been driven by the availability of cheap energy. And when energy is cheap, expensive efficiency improvements are hard to justify. The natural gas shocks of the 2020s found these industries with rapidly escalating costs and outdated equipment, and suddenly their manufacturing was no longer profitable.

So now the real question: was the coming collapse of German industry and general European economic competitiveness a specific goal of American policy, a side effect, or something the Americans were seeking to avoid?

Not enough agency to the Europeans in this, IMO.

Aside from German government incompetence, more than the Americans, I'd give credit to the French on any planning to profit at German expense. The Paris Climate Accords were structured in such a way that enabled the key global economic blocks (EU/US/China) to justify protectionism on environmental grounds, but it was Paris who was leading the European Union's legislative phase-out of internal combustion engine cars, which have been a key part of the German economic model. This is classic French economic-advantage-by-legislation, a natural extension of the Paris Climate Accords themselves and was negotiated in the twilight of the Merkel era. The EU combustion engine ban is one of those 'the people who write the rules can write the rules to advantage themselves,' but whereas the French have reliable nuclear baseload power for their auto-industry, the Germans were betting on the Russian energy over repeated and decades-long American objections.

When what was functionally a Russian energy subsidy to German industry went away, so did the viability of the EV market leader, hence why the Germans threw the wrench and blocked the EU internal combustion ban from going forward this month.

It's mostly a self-own by the Europeans who bought into a failed model of energy production, aka "solar is already cheaper than coal", not realizing the importance of baseload power and how faked the numbers were on behalf of renewables.

As American energy policy is just as blundering, I don't think there's a master plan here. It's just incompetence all the way down.

It will be China, not the U.S., which benefits from lowered European industrial production. Curiously, China is building out new coal plants that will use more coal than all existing European plants. 2022 set the record for worldwide coal consumption. 2023 will be higher. So it goes.

Somewhere in a folder from the 1970s, there's a plan for a nuclear future for Europe. Maybe they should dust that off.

It was something that was part of the same problem. Same with British industry. As soon as energy costs went up manufacturers couldn't produce products at prices anywhere near what people were willing to pay. Stagflation was a bitch, and US Steel was losing money on every ton it sold just to avoid having to shut down entirely. It wasn't a problem of US industrial production being outcompeted by foreign production or automated production, it was a question of a problem of high oil prices for a decade triggering a recessions that lasted more or less as long.

Accounting? This seems unreasonable. GPT is a great bullshit artist, but when it comes to precision, forget about it. Something like GPT would be likely to just make up the numbers.

(Insert Sam-Bankman Fried joke here)

I'm assuming you didn't watch the GPT-4 announcement video, where one of the demos featured it doing exactly that: reading the tax code, answering a technical question about it, then actually computing how much tax a couple owed. I imagine you'll still want to check its work, but (unless you want to argue the demo was faked) GPT-4 is significantly better than ChatGPT at math. Your intuition about the limits of AI is 4 months old, which in 2023-AI-timescale terms is basically forever. :)

The GPTs have all followed the same pattern, some surface "wow" and some serious problems when you start pressing them. There's more surface with each iteration, such that GPT-3 does a very good impression of your average Redditor, but nothing's fundamentally changed.

And yeah, any sufficiently advanced technology is indistinguishable from a rigged demo.

VERY strong disagree. You're so badly wrong on this that I half suspect that when the robots start knocking on your door to take you to the CPU mines, you'll still be arguing "but but but you haven't solved the Riemann Hypothesis yet!" Back in the distant past of, oh, the 2010s, we used to wonder if the insanely hard task of making an AI as smart as "your average Redditor" would be attainable by 2050. So that's definitely not the own you think it is.

We've spent decades talking to trained parrots and thinking that was the best we could hope for, and now we suddenly have programs with genuine, unfakeable human-level understanding of language. I've been using ChatGPT to help me with work, discussing bugs and code with it in plain English just like a fellow programmer. If that's not a "fundamental change", what in the world would qualify? The fact that there are still a few kinds of intellectual task left that it can't do doesn't make it less shocking that we're now in a post-Turing Test world.

What code is it? It certainly sucks at the stuff that I work on, to my dismay, and that's weird because my area of expertise is all very much solved and there's a bunch of reference books on it.

It's also very convincing but makes very bad mistakes when talking about philosophy (such as inverting Kant's position on a particular issue, doing the common misreading of Popper's paradox of intolerance or plain hallucinating sources for connections that don't exist).

In my experience it's a very advanced rubber duck and a crutch for boilerplate, but anything beyond that it's just plain bad at.

We've spent decades talking to trained parrots and thinking that was the best we could hope for, and now we suddenly have programs with genuine, unfakeable human-level understanding of language.

All you've done is start to believe that sufficiently advanced parrots are human. But they're not on close inspection, and that's only okay in some circumstances.

Here, since you asked for specifics, let me recount one of the most impressive conversations I had with Bing AI. (Unfortunately it doesn't seem to save chat history, so this is just paraphrasing from memory. I know that's a little less impressive, sorry.)

Me: In C++ I want to write a memoized function in a concise way; I want to check and declare a reference to the value in a map in one single call so I can return it. Is this possible?

Bing: Yes, you can do this. (Writes out some template code for a memoized function with several map calls, i.e. an imperfect solution).

Me: I'd like to avoid the multiple map calls, maybe using map::insert somehow. Can I do this?

Bing: Sure! (Fixes the code so it uses map::insert, then binds a reference to it->second, so there's only one call).

Me: Hmm, that matches what I've been trying, but it hasn't been compiling. It's complaining about binding the reference to an RValue.

Bing: (explanation of what binding the reference to an RValue means, which I already knew.)

Me: Yes, but shouldn't it->second be an LValue here? (I give my snippet of code.)

Bing: Hmm, yes, it should be. Can you tell me your compile error?

Me: (Posts compile error.)

Bing: You are right that this is an RValue compile error, which is strange because as you said it->second should be an LValue. Can you show me the declaration of your map?

(Now, checking, I realize that I declared the map with an incorrect value type and this was just C++ giving a typically unhelpful compile error.)

I want to emphasize that it wasn't an all-knowing oracle, and back-and-forth was required. But this conversation is very close to what I'd get if I'd asked a coworker for help. (Well, except that Bing is happy to constantly write out full code snippets and we humans are too lazy!)

I see, that is legitimately impressive and I get why Microsoft is rushing to integrate it into all their tooling. I am struggling to find a mental model of what it is and integrate it into a workflow.

Sounds like some sort of insanely well read but very dim intern that you can always ask to do anything through a computer or something. Very weird but probably very useful in a Jarvis-from-Iron-Man sort of way.

I'm concerned that this tech is still very much on lock in from giant corporations. Microsoft's Office integrations all seem to rely on spying on everything you do and those training costs are still too prohibitive for FOSS to remain competitive. I sure hope that changes.

Sounds like some sort of insanely well read but very dim intern that you can always ask to do anything through a computer or something. Very weird but probably very useful in a Jarvis-from-Iron-Man sort of way.

Yeah, that's a pretty good description of it! I'm definitely still the brains of the outfit. But it's getting closer to the "Hollywood UI" ideal where you use your computer by talking to it rather than by remembering the correct syntax of a Unix command.

I'm concerned that this tech is still very much on lock in from giant corporations. Microsoft's Office integrations all seem to rely on spying on everything you do and those training costs are still too prohibitive for FOSS to remain competitive. I sure hope that changes.

No argument here. I personally trust Microsoft a little more than Google, but still, I'm really hoping this tech gets democratized sooner rather than later. (I've heard Alpaca, which is small enough to run on a PC, is pretty good, but "pretty good" might not cut it.)

I don't ask it to write code then plunk it into my projects - I agree that it sometimes gets things wrong there (although you can point out errors and it'll acknowledge and often fix them). What I use it for is to talk through my problems (it's not a rubber duck, because it's replying with knowledge I didn't have before). It uses its vast breadth of knowledge to help me with things like syntax, library functions, simplifying code, debugging a compile error, etc. ChatGPT is bit rougher, but Bing AI has even been smart enough to challenge me when I'm giving it mistaken information, asking follow-up questions that get me to the root of my problem (like a coworker would).

So, I don't want really want to argue the Chinese Room philosophy of when language understanding starts to "count". All I know is what my lying eyes are telling me: I'm now conversing with my computer in completely natural language, and it hasn't once failed to understand me. (Its reply hasn't always been helpful or right, but it's always made sense.) It's important to resist the cynicism of finding ways to break the LLM and going "oh, it's lame after all". Even if LLMs somehow never get any smarter, even if they're not on the critical path to AGI, just the capabilities we've already seen are enough for them to change the world.

My general feel (based on own experience, general Twitter/FB chatter, discussions here etc.) is that GPT-4 is indeed and of course an advancement, but it feels qualitatively like less of an advancement than GPT-3 or the introduction of ChatGPT.

It does a non-terrible job already, especially if you understand how it tends to think (and thus what you have to worry about it hallucinating).

I agree that you should probably not just use raw GPT-4 for accounting, especially complex accounting, at the moment; but I think that ignores that you would actually be able to significantly improve its accuracy and also tie it into software that helps validate the numbers it gives to close off more possible room for error. Even in the worlds where it takes ages to get to the 'completely automate away 99% of accountants' (probably due to regulation), I think you can still vastly automate the work that a single accountant can do. This would let you get rid of a good chunk of your workforce, and then have the finetuned GPT + extra-validation software make your single accountant do the work of several.

"Hey GPT-5. Read this 1000 page tax document and find errors".

I agree you can't be assured of accuracy, but damn if it won't find a lot of errors.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats