site banner

Friday Fun Thread for January 5, 2024

Be advised: this thread is not for serious in-depth discussion of weighty topics (we have a link for that), this thread is not for anything Culture War related. This thread is for Fun. You got jokes? Share 'em. You got silly questions? Ask 'em.

1
Jump in the discussion.

No email address required.

I just spent half an hour doing research and napkin math about WWII naval vs. aerial bombardment. This was related to suggestion for a Hearts of Iron mod. Partway through the ensuing discussion, one of the devs steps in with his own estimates. They are based on some flawed math, but more importantly, they are a screenshot from Google Bard.

Observation one: it is absolutely insane that you can give a computer word problems and have it spit out formatted, plausible answers, complete with hypotheticals. There were caveats about how the guns were never designed for the proposed use and a table of how the answer would change with lower rates.

Observation two: it is completely insane that you can do this and have the computer lie to you. Not with any malice! But it will give you a wrong, even incoherent answer with the exact same confidence as a correct one. Those symbols get strung together all the time in its training data, after all.

Observation three: well, the third type of insanity ought to go unremarked. I’m not upset that the dev leaned on this AI. I got the impression he was just tossing in his two cents, not defending the position. It does raise the question—

Is it possible to raise the general level of skepticism about AI answers, rather than AI technology?

Even AI Evangelists do not take AI answers at face value (at least if they're even mildly informed about the technology). That is a bad idea right now, and will be until the hallucination rate drops further. For anything non-trivial, such as medical advice, I would highly recommend at least generating multiple responses, or following any links and citations the old fashioned way to sniff check them.

Of course, the worst sin this dev committed was to use Bard, it's still noticeably inferior to GPT-4. The latter is free through Microsoft Copilot, why use Google's shitty alternative?