site banner
naraburns  nihil supernum  1yr ago (text post)   3314 thread views

Friday Fun Thread for March 3, 2023

Be advised: this thread is not for serious in-depth discussion of weighty topics (we have a link for that), this thread is not for anything Culture War related. This thread is for Fun. You got jokes? Share 'em. You got silly questions? Ask 'em.

4
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

A few months ago OpenAI dropped their API price, from $0.06/1000 tokens for their best model, to $0.02/1000 tokens. This week, the company released their ChatGPT API which uses their "gpt-3.5-turbo" model, apparently the best one yet, for the price of $0.002/1000 tokens. Yes, an order of magnitude cheaper. I don't quite understand the pricing, and OpenAI themselves say: "Because gpt-3.5-turbo performs at a similar capability to text-davinci-003 but at 10% the price per token, we recommend gpt-3.5-turbo for most use cases." In less than a year, the OpenAI models have not only improved, but become 30 times cheaper. What does this mean?

A human thinks at roughly 800 words per minute. We could debate this all day, but it won’t really effect the math. A word is about 1.33 tokens. This means that a human, working diligently 40 hour weeks for a year, fully engaged, could produce about: 52 * 40 * 60 * 800 * 1.33 = 132 million tokens per year of thought. This would cost $264 out of ChatGPT.

https://old.reddit.com/r/singularity/comments/11fn0td/the_implications_of_chatgpts_api_cost/

...or about $0.13 per hour. Yes technically it overlooks the fact that OpenAI charge for both input and output tokens, but this is still cheap and the line is trending downwards.

Full time minimum wage is ~$20k/year. GPT-3.5-turbo is 100x cheaper and vastly outperforms the average minimum wage worker at certain tasks. I dunno, this just feels crazy. And no, I wont apologize for AI posting. It is simply the most interesting thing happening right now.

Not Fun thread material, fit for a separate post.

This week, the company released their ChatGPT API which uses their "gpt-3.5-turbo" model, apparently the best one yet, for the price of $0.002/1000 tokens

Well, as you quote, they imply it's not the best one yet, at least in «some use cases», and some experiments show it indeed is intellectually inferior to the most recent naive GPT-3 version; maybe this is inherent to ChatGPT model we see in the demo, because it is known that RLHF can give models the computational equivalent of brain damage as far as benchmarks are concerned, maybe it's something specific for the new one. I am not sure how they've achieved the price cut (though my intuition is that this is cynical undercutting at a loss to nip competition in the bud and keep the data flywheel accelerating) – perhaps it's smaller (trained from scratch, distilled, etc.), or aggressively quantized, or more greedily sampled, or they've somehow increased the batch size into high thousands, whatever. In any case, Altman is a great showman and this may not be the revolution it seems like at the moment. Do we really need an endless stupidity generator with short context? Most people who are on this level (and I do think we can now meaningfully say that some people are on this level) around ChatGPT) aren't exactly making money with their smarts. It's nice for employers to automate even more drudgery, of course.

But in principle, yes, optimizations and next-gen models with more novel architectures than Vawani!Transformer will definitely allow even cheaper inference (I think GPT-4 under the hood of Bing already is that).

I suspect it, too, is a gimped/lossily accelerated version, with the full one to be released under the «DV» brand as part of their Foundry product. I may be wrong, however, and ultimately this PR-driven nomenclature is making less and less sense, they have trained like a dozen major models at this point, and more finetunes. Same story with Google's PaLM.

More importantly, quantity has a quality all of its own. DV offers a context window of 32k tokens. If the model can be cheaply ran with such enormous contexts, this will more than compensate for some intellectual deficiency in the context-sparse text-prediction mode. You have seen effects of StableDiffusion mega-detailed prompting, and of prompt prefixes that amount to a single page – now imagine 20 pages of precise specification, few-shot examples, chain-of-thought induction, explicit caveats for every failure mode discovered in testing; basically a full employee's manual. Writing and testing these megaprompts may become a new job for ex-middle managers who have suddenly transformed into leaf nodes of the org chart – for a short while.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats