faul_sname
Fuck around once, find out once. Do it again, now it's science.
No bio...
User ID: 884
faul_sname
Fuck around once, find out once. Do it again, now it's science.
3hr ago
Agreed.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
10hr ago
Specifically in the field of medicine, they can do most of the raw cognitive labor doctors do - as well or better than the average doctor.
As an outsider, I am unsure of how impressive this is. I know that "most of the raw cognitive labor programmers do while writing code" is fairly rote, but I don't know how true that is for doctors.
How much of the raw cognitive labor doctors do could be done by a bright undergrad with access to uptodate and a bunch of case histories, both with semantic search?
faul_sname
Fuck around once, find out once. Do it again, now it's science.
1d ago
I've actively experimented with this, and it's one of my vibes benchmarks for any new model release.
Of course, my standards were that it should pass my own sniff test. That roughly amounts to "does that sound like me?" and "are those the kinds of arguments I'd make?". They do a decent job, and have for a while. Not perfect, but good enough to fool most of the people, most of the time.
Huh. It's been my sniff test for new models as well, and so far I have not seen much success. It should be easy! This is literally the most LLM-flavored task to ever task! And yet. I've sunk probably 50 hours into it.
My most recent attempt, which I sunk about 10 hours and $100 into, and which got a lot closer than any previous attempts, involved giving Claude a corpus of all my past writing and having it try multiple different ways of producing text on arbitrary topics in my voice. The things tried were
- Just throw a lot of writing samples and ask it to write in the same voice (just sounded like standard generic Claude)
- Take 5 of my writing samples, come up with plausible prompts to generate them, throw them in llama-405b base (hyperbolic) in format [>prompt>...>/prompt>>response>my_sample>/response>] x5 followed by >prompt>[the real prompt>/prompt>>response> (didn't follow prompt, broke with my writing style fairly early)
- That, but doing a product-of-experts thing with multiple continuations (same result, if anything a little worse)
- Standard SFT on my voice (gets the texture of my writing right, but can't maintain coherence for more than a sentence or two, if trained for more epochs just memorizes the things I've written and ignores the prompts)
- Took a bunch of my writing samples, flattened them by "rewriting them to sound better", SFT on task of reversing that i.e. "here is an AI generated passage [slopified original]. Rewrite it in faul_sname's voice: [original passage I wrote]" (kinda sounds like me if I was actively having a stroke)
- Same but DPO instead of SFT (different kind of stroke)
- Clever-sounding GAN setup (couldn't get it working, gave up after a few hours)
On the one hand, I was very impressed by how good Claude was at running a whole bunch of these experiments very quickly. On the other hand, it did not work for me, not even at the level of "passes the sniff test", much less at the level of "standard stylometry techniques say it sounds like me".
[A corpus of all my past writing] would be an absolute pain to collate, both for me and for Claude
I think you'll find that this is one of the tasks that is now much much easier. It's actually been within the capabilities of frontier models since Sonnet 4.0 (which is when I went ahead and gathered said corpus, on the theory that it'd be pretty useful to have). The prompt you're looking for is something like "Here's a chrome instance running with --remote-debugging-port and logged in on most of the sites I post on with a tab open for each. Go generate a corpus of all my publicly available writing".
Would you be willing to pay for that or provide access
Yeah. An H100 for 24h would run in the ballpark of $40, well worth it for me to provide. Vast allows transferring credits from one account to another, so I'd happily just transfer $50 of credits over if someone actually wants to do this. Does seem like rather a lot of work though.
We'd need to decide if I'm allowed to use LLMs myself, in the same manner I already do. Claude wouldn't get the benefit of me giving it feedback or editing its output in any capacity
Yeah, that's entirely reasonable. Your voice is very different from Claude's voice.
Do you really need us for the test, beyond whatever blinding or scraping of my writing is necessary?
Yeah, I'm hoping you can prove me wrong here. I've been trying to do this since back in late 2019 when nostalgebraist-autoresponder was shiny and new. I want a good simulacrum of myself! I want to have that simulacrum, and I want to loom it. I want to build an exobrain, and merge with it, and fork off a copy running in the cloud.
BTW I expect there's a substantial market for anyone who manages to build this in a repeatable way. I've looked, and there are as of now no commercial offerings for this (though there are a few commercial offerings that pretend to be this).
I'd struggle to pin you as a moderate either, though I can't recall you expressing strong opinions that aren't more research/alignment oriented. If my hunch is true, then you aren't the target audience for this! Plus I'll eat my hat if you don't already have access to the best models out there.
I only have access to the models you can obtain access to with money - I expect I'm 3-6 months behind the best of what insiders at Anthropic or OAI have access to.
I don't think I'd describe you as an LLM skeptic
An LLM skeptic is an LLM idealist who's been disappointed :)
Also, this is slightly off-topic. We're not evaluating the agents on their ability to write code, we're testing their ability to mimic me convincingly.
I expect looking like you stylometrically while also exhibiting the same patterns of thought you exhibit on a specific topic will involve writing code. But code in the service of trying to mimic you convincingly, rather than in the service of producing some specific durable software artifact.
For the record, I do expect this to be within the capability window within the next 18 months, but I would be pretty surprised if you managed to get Opus 4.6 specifically to do it.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
1d ago
What do you think Claude cannot do?
Write a coherent thousand word post in your voice about a topic of your choosing sufficiently well to fool standard stylometry techniques, and pass the sniff test as sounding like you to others here, even given
- Access to all of your public past writing
- Access to base models
- Access to fine-tune base or instruct models
- Access to a vast.ai box with an H100 for 24 hours to do whatever else it wants
faul_sname
Fuck around once, find out once. Do it again, now it's science.
4d ago
What the fuck did I just watch?
faul_sname
Fuck around once, find out once. Do it again, now it's science.
4d ago
$623/mo on food at home plus another $393/mo away from home. Even the <15000 group spends $625/mo across all food, $416 of which is groceries, so I stand by $800/mo not being extravagant if you cook all your meals. Probably a bit of room to budget but not that much.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
5d ago
If you're eating most of your meals at home, that's about $3 per person per meal. You can eat reasonably well on that, but it doesn't seem exorbitant. I spend about twice that for a family of 3 (because groceries are approximately free compared to rent and taxes, so why not optimize for quality rather than price).
I think the key to getting good results is figuring out how to get a verifiable success/failure signal back into the LLM's inputs. If you've got an on-premise application and as such have no access to logs and such from the customer, I expect the place you'll see the most value is a prompt which is approximately "given [vague bug report from the user], come up with a few informed hypotheses for what it could be by looking at the codebase, and then, for each hypothesis (and optionally "and also my pet hypothesis of XYZ" if you have a hypothesis) , iteratively create a script which would reproduce the bug on this local instance of the stack if the hypothesis were correct [details of local instance]".
As an added bonus, the code to repro a bug is hard to generate but easy to verify, and generally nothing is being built on top of it so if the LLM chooses bad or weird abstractions it doesn't really matter.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
11d ago
If you're represented by a lawyer, why are you asking an LLM questions about your case. That's the point of having a lawyer! If the lawyer can't explain things sufficiently, then you simply don't have a good lawyer.
I expect many people do not have a good lawyer, by this standard. Or can't afford a good lawyer, for the number of hours of the lawyer's time they'd need to actually understand what's going on with their case and what that means in practical terms for them.
Any email sent from the attorney's account is going to be encrypted, and they should ensure that any email the client sends will be encrypted as well.
To your point, this is probably the real answer. If the logs of the chat don't exist, they're not going to show up in court.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
12d ago
As far as I can tell
- Google didn't actually get sanctioned for that
- The only documents Google actually had to provide were the ones where the attorney did not say literally anything at all in the thread
- And only about 80% of those, even
I agree with your expectation that judges would not like that product very much.
Well yes. I expect this is mostly bottlenecked on lawyers, and on reflection I mostly expect that it'd be a white label "AI boosted chat with lawyer" product that law firms could offer rather than "the AI lawyer company". Maybe combined with inbound lead generation in the form of a directory of firms that offer the service.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
12d ago
New case law just dropped[^1]: a guy was charged with a $300M securities fraud. Before his arrest he used Claude (the consumer product) to research his own legal situation. He then handed the outputs to his defense counsel and claimed attorney-client privilege. The prosecutor said "no, that's not how this works, that's not how any of this works", and the judge agreed[^2]. That means that as of this decision, precedent says that if you go to chatgpt dot com and say "hey chatgpt, give me some" legal advice, that's not covered under attorney-client privilege.
On the one hand, duh. On the other hand, it really feels like there should be a way to use LLMs as part of the process of scalably getting legal advice from an actual attorney while retaining attorney-client privilege.
I expect there's an enormous market for "chat with an AI in a way that preserves attorney-client privilege", and as far as I can tell it doesn't exist.
It was also interesting to read the specific reasoning given for why attorney-client privilege was not in play:
The AI-generated documents fail each element of the attorney-client privilege. They are not communications between the defendant and an attorney. They were not made for the purpose of obtaining legal advice. And they are not confidential. Each deficiency independently defeats the defendant's privilege claim.
I notice that none of these reasons are "conversations with AI are never covered by attorney-client privilege." They're all mechanical reasons why this particular way of using an AI doesn't qualify. Specifically:
- Claude is not an attorney, therefore Claude is not your attorney, therefore this wasn't communications between a defendany and their attorney.
- Sending a document to your lawyer after you create it does not retroactively change the purpose that the document was created for.
- Anthropic's consumer TOS says they can train on your conversations and disclose them to governmental authorities, and so the communications are not confidential.[^3]
The prosecutor also argues that feeding what your attorney told you into your personal Claude instance waives attorney-client privilege on those communications too. If a court were to agree with that theory, it would mean that asking your LLM of choice "explain to me what my lawyer is saying" is not protected by default under attorney-client privilege. That would be a really scary precedent.[^4]
Anyway, I expect there's a significant market for "ask legal questions to an LLM in a way that is covered by attorney-client privilege", so the obvious questions I had at this point were:
- Is there an existing company that already does this, and are they public / are they looking to hire a mediocre software developer?
- If not, what would it take to build one?
For question 1, I think the answer is "no" - a cursory google search[^5] mostly shows SEO spam from
- Harvey, which as far as I can tell from their landing page is tools for lawyers (main value prop seems to be making discovery less painful)
- Spellbook AI (something about contracts?)
- GC AI, which... I read their landing page, and I'm still not sure what they actually do. They advertise "knowledge base capabilities" of "Organize", "Context", "Share", and "Exact", and reading that page left me with no more actual idea of their business model than before I went there.
- Legora has a product named "Portal" which describes itself as "A collaborative platform that lets firms securely share work, exchange documents, and collaborate with clients in a seamless, branded experience." but seems to be just a splash screen funneling you to the "book a demo" button.
So then the question is "why doesn't this exist" - it seems like it should be buildable. Engineering-wise it is pretty trivial. It's not quite "vibe code it in a weekend" level, but it's not much beyond that either.
After some back-and-forth with Claude, I am under the impression that the binding constraints are
- The chat needs to be started by the attorney, rather than the client - Under the Kovel doctrine [^6], privilege extends to non-lawyer experts only when the attorney engages them
- The agreement with LLM providers commits to zero training and no voluntary disclosure to authorities (pretty much all the major LLM providers offer this to enterprise customers AFAICT)
- It needs some way of ensuring that the chats are only used for the purposes of getting legal guidance on the privileged matter
None of these seem insurmountable to me. I'm picturing a workflow like
- Client signs up for an account
- Client is presented with a list of available lawyers, with specialties
- Client chooses one
- That lawyer gets a ping, can choose to accept for an initial consultation about a matter
- Lawyer has a button which opens a privileged group chat context between them, the LLM, and the client about that matter
- Lawyer clicks said button.
- In the created chat, the client can ask the LLM to explain legal terminology, help organize facts or documents the lawyer requested, or clarify what the lawyer said in plain language, or do anything else a paralegal or translator could do under the attorney's direction.
Anyone with a legal background want to chime in about whether this is a thing which could exist? (cc @faceh in particular, my mental model of you has both interest and expertise in this topic)
[^1]: [United States v. Heppner, No. 25 Cr. 503 (JSR) (S.D.N.Y. Feb. 6, 2026)] (https://storage.courtlistener.com/recap/gov.uscourts.nysd.652138/gov.uscourts.nysd.652138.22.0.pdf). The motion is well-written and worth reading in full.
[^2]: Ruled from the bench on Feb 10, 2026: "I'm not seeing remotely any basis for any claim of attorney-client privilege." No written opinion yet.
[^3]: This argument feels flimsy, since attorneys send privileged communications through Gmail every day, and Google can and regularly does access email content server-side for reasons other than directly complying with a subpoena (e.g. for spam detection). It could be that the bit in Anthropic's TOS which says that they may train on or voluntarily disclose your chat contents to government authorities is load-bearing, which might mean that Claude could only be used for this product under the commercial terms, which don't allow training on or voluntary disclosure of customer data. I'm not sure how much weight this particular leg even carried, since Rakoff's bench ruling seems to have leaned harder on "Claude isn't your attorney."
[^4]: A cursory search didn't tell me whether the judge specifically endorsed this theory in the bench ruling. So I don't know if it is a very scary precedent, or just would be a really scary precedent.
[^5]: This may be a skill issue - I am not confident that my search would have uncovered anything even if it existed, because every search term I tried was drowned in SEO spam.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
18d ago
... now you're just threatening me with a good time. Can we advocate that the states ignore the FDA too while we're here?
faul_sname
Fuck around once, find out once. Do it again, now it's science.
18d ago
This sets up some pretty fucked incentives. Setting up fucked incentives has historically not gone well.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
20d ago
And the guy behind ClawdBot / MoltBook (or whatever its called now) has openly discussed how his own deployment of ClawdBot was thinking and executing ahead of him.
I will point out that MoltBook had exposed it's entire production database for both reads and writes to anyone who had an API key (paywalled link, hn discussion).
And this is fairly representative of my experience with AI code on substantial new projects as well. In the process of building something, whether it's something new or something legacy, the builder will need to make thousands of tiny decisions. For a human builder, the quality of those decisions will generally be quite tightly correlated to how difficult it is for a different human to make a good decision there, and so, for the most part, if you see signs of high-thoughtfulness polish in a few different part of a human-built application that usually means that the human builder put at least some thought into all the parts of that application. Not so for "AI agents" though. One part might have a genuinely novel data structure which is a perfect fit for the needs of the project and then another part might ship all your API keys to the client or build a SQL query through string concatenation or drop and recreate tables any time a schema migration needs to happen.
That's not to say the "AI coding agent" tools are useless. I use them every day, and mostly on a janky legacy codebase at that. They're excellent for most tasks where success is difficult or time-consuming to achieve but easy to evaluate - and that's quite a lot of tasks. e.g.
- Make an easy-to-understand regression test for a tricky bug: "User reports bug, expected behavior X, observed behavior Y. Here's the timestamped list of endpoints the user hit, all associated logs, and a local environment to play around in. Generate a hypothesis for what happened, then write a regression test which reproduces the bug by hitting the necessary subset of those endpoints in the correct order with plausible payloads. Iterate until you have reproduced the bug or falsified your hypothesis, If your hypothesis was falsified, generate a new hypothesis and try again up to 5 times. If your test successfully reproduces the bug, rewrite it with a focus on pedagogy - at each non-obvious step of setup, explain what that step of setup is doing and why it's necessary, and for each group of logically-connected assertions, group them together into an evocatively-named assert() method."
- Take a SQL query which returns a piece of information about one user by id and rewrite it to performantly return that information for all users in a list
- Review pull requests to identify which areas would really benefit from tests and don't currently have them
- Review pull requests to identify obvious bugs
faul_sname
Fuck around once, find out once. Do it again, now it's science.
21d ago
I think about this a lot, but I also catch myself thinking about how easy it must have been in the 90s to find alpha in X, and then realize that with the knowledge I have now it would be easy, but that obtaining that not-yet-common knowledge would have been much harder in the 90s. I'm sure that there's similar alpha available today if you know where to look for it, but if it was easy to find, it wouldn't be alpha.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
21d ago
Even Disney World has MBA'd itself into a place I would no longer remotely describe as the "happiest place on earth".
I actually went to Disneyland with my wife and daughter a couple months ago, and I was shocked by how much it wasn't MBA'd. The tickets were cheaper (inflation-adjusted) than they were when I was a kid, the food was decently good and not horribly expensive (~$20 / meal for decent bbq with big enough portions that we only needed one full meal plus a few snacks during our entire time from park open to park close), there weren't really any of the rigged carnival games that are optimized to make it seem like you just barely missed the big prize and should just try One More Time that you see in other amusement parks, and the lines didn't shove ads in your face (again, unlike other amusement parks). Possibly I just went in with sufficiently low expectations that I was pleasantly surprised.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
27d ago
Plus Trump's team is already running into random vigilante judges in farflung circuit courts attempting to adjust whatever they pass.
I think this is a symptom of the things where the legislative branch refuses to legislate, leading to a power vacuum which both the executive and the judiciary branches try to fill.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
27d ago
Take the statement "I think ICE was in the right during the recent shooting, because <reason>".
Take X, and plug it into the statement "I think we should go down the list of registered Democratic voters and send hit squads to their houses to kill everyone present, because <reason>".
Does the sentence still make sense?
Example A: <reason>="because declining to enforce laws if bystander-activists actively make the situation more dangerous sets up terrible incentives". "Kill the dems" statement makes no sense if you put this reason in, thus it is not an example of the sort of thing DiscourseMagnus was talking about.
Example B: <reason>="because the dems had it coming for ruining our country". "Kill the dems" statement does make sense if you put this reason in, thus it is an example of the sort of thing DiscourseMagnus was talking about.
TBH on here I don't see much of example B. On xitter I do, but the discourse here on the motte has been refreshingly free of that for the most part. I do agree with DiscourseMagnus that example B is bad and the sort of thing I want to see less of, but I don't agree with his implication that it's the sort of thing I see a lot of here.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
27d ago
This really seems like a case where you should petition your elected representatives to change the laws. If our legislators actually started legislating that would help a lot with the current power struggles between the judicial and executive branches, and maybe having their constituents getting on their case for failing to legislate would help with that.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
27d ago
Midterm elections are in 9 months. One way to lose is by declining to try, but another way to lose is deciding to try really hard, fucking everything up badly in a highly legible way, and being booted out of your position.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
28d ago
It's about a 30 min walk / 10 min uber from rockridge bart, so pretty doable. There's a sequences reading group every Tuesday at 6:30 pm at lightcone if you want to get the full bayrat experience (cc @falling-star).
faul_sname
Fuck around once, find out once. Do it again, now it's science.
29d ago
The most sobering part? It’s domestic. Funded, trained (somewhere), and directed by people who live in the same country they’re trying to paralyze law enforcement in
Pangram says 100% AI generated. Make what assessments you will about the reliability of the author and how likely it is that they're actually a former Special Forces Warrant Officer.
faul_sname
Fuck around once, find out once. Do it again, now it's science.
1mo ago
But the rest of the rhyme is correct.
Gemini's sample is impressive! Color me impressed, especially that a straight-up prompt produced that (though I suppose if any technique would get it with current models, it'd be "one shotting through a prompt" rather than "iterative refinement towards a target").
It doesn't sound quite the same as the version of you that lives in my head, but it's awfully close. E.g. I can't imagine you saying
since you don't tend to drop spurious technical details into your walls of text unless they serve a purpose (and also because I half suspect you're not a fan of the amyloid theory of alzheimers). More generally, the Gemini piece has a higher density of eyeball kicks than I model your writing as having. And I model your writing as having a lot of those, for a human.
It also seems to drift away from your voice in the second half. And it fails the stylometry vibe check - Pangram detects AI with medium confidence - but maybe in a way that's reparable. And actual stylometry (cohens d of +17 on dashes, +2 on words >9 letters, +1.5 on mean word length in general, -2 on 3-4 letter words, -1.2 on punctuation in general - i.e. you use more and more varied punctuation and shorter words, by a notable margin, and Gemini uses way, way, way more dashes). Still, it's much much better than I expected! (and yeah, the Claude one is not even worth discussing)
Interestingly, your results look much, much better to me than the ones I get myself. I ran the same test as you did against Gemini, and got these not-very-good attempts: 1 2 3. Gemini took distinctive phrases (e.g. "85% agree") and ideas (e.g. "claude code as supply chain risk") I have used once in the corpus, fixated on them, and stitched them together into a skinsuit which superficially resembles my writing but doesn't hold up under scrutiny. Interestingly, that's a very base model flavored failure mode. I have grown unused to seeing base-model-flavored failure modes, and as such Gemini is much more interesting to me now.
More options
Context Copy link