BANNED USER: temperamental dysregulation
DaseindustriesLtd
late version of a small language model
Tell me about it.
User ID: 745
Banned by: @Amadan
DaseindustriesLtd
late version of a small language model
3yr ago
Does your theory predict collapse in immigration once AI-powered automation makes this (already net budget negative over lifetime, in many cases) addition clearly counterproductive?
How's this for drama: the people supporting mass immigration are driven by basically moral considerations (though their morality may be different from what we believe), and would rather slow down productivity growth than allow their clients to be made patenly uneconomical for "capitalism".
First we have to make sure we both have the same thing in mind when we're talking about qualia.
Obviously this is a doomed enterprise. I agree with you that, by this standard of skepticism, it is impossible to establish that someone has a mind at all. In practice I really do not believe that your mind is much similar to my own, so there isn't really a reason to believe that you can conceive of qualia in a way that I'd endorse, not matter what you type, and vice versa.
A quale just is your first-person experience of a sensation. It's the actual pain you feel when you feel, well,
You are going in circles. This is worse than circular, this is contentless. "A qualia just is a first-person experience". Okay? "It's the actual pain you feel". Sure, and? The question is whether "a brain" "feels" "pain", in some way that can be operationalized.
If I were you, I'd try mindwipe myself at this point, because it's clear that the procedure you're using for reasoning has run into its inherent ceiling.
Is there any independent information content to a qualia of red, something that actually distinguishes it from other quale, and something that is not insistence on it being an ineffable experience that is "first-person"? (Information: "what is conveyed or represented by a particular arrangement or sequence of things.") Well, whatever that content is, it is isomorphic to the working bits of the corresponding brain state. The brain state contains that information, we know very well that it does. Maybe you can't be sure that you observe its traces "directly" on fMRI, but in theory it exists necessarily, so long as physical universe works by physical rules.
So the question is: can information (of a brain state, or perhaps any information encoded on any physical substrate) "feel like anything" for the identity to be valid. I do not see a well-formed argument for why it cannot, sans some talk about conceivability.
But that is just a mindless token stream. I straight up do not recognize that you have the capacity to evaluate whether this is conceivable or coherent.
And that's that I guess.
You could be hallucinating the whole universe, for example, and no one is having any actual conscious experience except you. Saying that you can directly observe qualia strikes me as akin to saying that you can directly observe that e.g. the apple in front of you actually exists in reality and isn't just a hallucination or a simulation
Irrelevant. The p-zombie thought experiment assumes physicalism, which means it assumes the conventional model where brains contain information.
There really is strictly nothing to your position than the exasperated feeling that there must be some honorable, justifiable escape from physicalism, that it must have a flaw, that it must not explain the whole picture, that there must be a "conceivable" way for brain states to somehow "not feel like anything to themselves" as they exchange information on their own plane of existence. But I maintain that A) this idea is not coherent and B) you are vendor-locked from stress-testing it because tokens like "I", "you" and "feel" are privileged in your design and you won't be able to look at the problem without them no matter how much I ask you to.
I've learned not to argue with LLMs and I stop arguing with you.
Both are functions that map inputs to outputs; there's no principled distinction you can draw between them.
The distinction is that I do not think «maps inputs to outputs» is the most relevant way to understand the former. There is nothing outside the true generative function of our universe (let's assume it really is the universal wavefunction) that it addresses, no heap of states. It is not an arbitrary "mapping", it is a self-contained entity (see Tegmark I guess); no part of state it describes is not generated by it, so the ordering of its states is unique and necessary. I think this, or something very close to this, is what makes a "universe", and so preconditions for "Universe X" to be coherent. Universe is absolute necessity.
In contrast, a table-governed «universe» is entirely made of slices that are not generated by any algorithm that is related to the table or that could impose order on them; thus they cannot have any particular order; thus, this is not a universe but a set of descriptions of states and an unrelated string that claims they have an assigned sequence, and it is incoherent to call this a universe.
We can do in-universe approximations of physical laws that have different properties and are «coherent» for utilitarian purposes, but it must be kept in mind that this is what they are. For a crayons universe, the equivalent is not the state table but the creative intent of Dan Harmon or what's his name. A crayons universe is coherent not as an actual self-contained universe but as a self-contained TV series episode.
I admit I might be very confused about this. Also, there can be even a coherent crayons-universe in this definition, at least it is imaginable that there can be.
I don't understand what "the physicalist model contains elements corresponding to a non-circular definition of quale" means
I think you are just denying physicalism at this point.
There are abnormal medical cases where we can observe that someone is having an experience, but they aren't aware of it.
No. Anosognosia was just a handy example of a differing information availability condition. Blindsight might be a better one. But specifically, I simply mean the delta between having sensation and perception. (I am not sure if discussing awareness or attention helps clarify this but they are discussed in this regard). A thing like "seeing red" is perception by canonical definition. We know how perception is implemented physically. Ergo we know one way for quale to work: it would be just for perceptions, implemented the way we know they are, to be informationally available to the mind.
you infer that qualia just is this brain activity that we've measured, that it's identical to it.
Yes, it is.
You infer furthermore that this inference is logically necessary
It is logically necessary due to definitions of all elements involved, perception, quale, brain and so on. Perceptions are definitionally identical with quale, and this is how perceptions are implemented in the brain mechanically, the whole causal chain. Once more, you can insist until you are blue in the face that they are not available to a non-material Cartesian soul and therefore some other, fancier quale are needed. But you absolutely do not get to say that a physical human can have none.
It is not logically necessary that Cartesian souls outside normal causality stream do not exist. It is not logically necessary that they would have access to perceptions in the brain. It is not incoherent to demand that they be afforded something else. It's just scholastics. The brain has access to perceptions in the brain; the zombie totally has its zombie-quale.
But, I claim, these inferences aren't logically necessary; that's the whole matter at issue here.
Unless you can object to the identity of perception and qualia with something more than A) "it's not logically necessary that they be identical, although I don't have a definition for qualia that makes them non-identical and isn't just insistence that there are non-physical quale" or B) "physicalism actually sucks though" , I maintain that it is in fact necessary.
because we can coherently imagine qualia and brain states coming apart. We can do this because I can't directly observe your qualia in the way that I can directly observe your brain states, your behavior, etc.
This is obviously circular. You insist that you can "coherently imagine" A since you can't observe B. But denying that you observe B does not automatically make it so you actually don't (again, there is such a thing as blindsight).
I don't even know what it would mean to observe someone else's qualia, even with a direct link between our brains.
You can observe them with fMRI. This is just a difference in observing them from the outside versus from the inside. This isn't a problem unique for quale: for a thing to exist, there must be difference between contents included in that thing and everything else. This is how all things work. Your perceptions are events of self-attention in your nervous system, not in mine.
That's why there's a Hard Problem in the first place.
Reasons «there is a Hard Problem» include: narcissism, mental sloppiness, obsession with gut feelings, excessive funding, insight porn industry, aftershocks of psychedelic revolution, and popularity of Coachella with some well-connected types suffering from all the above. But they have nothing to do with logic. The idea of Hard Problem is dumb, embarrassing on par with an illiterate stoner's fascination about whether he has a "hand" or "five fingers plus a palm".
To be honest I'm bored of this debate. It feels dirty to even think back to times I cared for it.
To be clear, I think we still don't have a principled reason to believe that this paradigm – in this vanilla form, autoregressive LLMs pretrained on text – can carry us to human level or beyond. It might be the case that LeCun is right and LLMs on their own are an off-ramp. It might run into diminishing returns or totally plateau any moment now; just because better «understanding» allows to make better predictions and we reinforce the latter doesn't mean we can get infinitely much of the former, any more than we can incentivize a human to run barefoot at 100 mph.
But people who seriously bought into such skepticism got caught off-guard by GPT-3 already.
And I expect amazing innovations like adding a backspace to keep the pretraining thesis viable far beyond GPT-4. The number of papers that propose improvements is absolutely mind-boggling, nobody keeps up with building deployable tech on those insights. People who follow the literature see the outline of AI of the near future and it's pretty damn formidable, much more than the progress in public demos and products can suggest.
DaseindustriesLtd
late version of a small language model
3yr ago
Tell me the exact ontological status of our laws of physics and how they "govern" our universe, and I'll tell you the exact ontological status of the state table
I don't think this statement has any content sans vacuous (the fact that you can reason in a similar manner about both).
Well, that was a mistake on his part, and I wouldn't offer that as a "definition".
On the contrary, I think that definition counts and yours are circular.
I think part of the disconnect here is that you're underestimating what a high bar it is to show that something is logically incoherent.
And I think you overestimate human aptitude at logical reasoning over sufficiently large sets of interdependent statements while watching out for incoherence. Also at recognizing which statements are relevant.
Because we're not doing "serious further theorizing"; we're arguing about the internal logical coherence of a concept.
That's probably fair.
Let me put it like this. I reject that P-zombie is only a «concept» and not a «model». I think the whole school of thought that allows to claim the opposite is illegitimate and I won't engage with it further.
The definition of p-zombie as a de facto physical human entails the entire baggage of physical theory and all its concepts. It's not some neat string like «human modulo quale» but that string plus our entire physicalist model of a human. The physicalist model contains elements corresponding to a non-circular definition of quale, thus a zombie can't not have quale; and the «concept» of p-zombie as a human modulo quale, situated in the proper context of dependencies of the word "human", is either incoherent or circular due to people insisting on non-physicality and saying these quale don't count and some others, which have an arbitrary relationship with our reality (might be epiphenomena, might be monads or whatever) must exist for non-zombie humans.
I take the main thrust to be something like, "it is a priori part of the concept of qualia that they play a causal role in our behavior", which would entail that p-zombies are incoherent.
No, I think this is just circular insistence on physicalism and not my argument. Physicalism taken seriously covers all of causality.
Can you elaborate on this concept?
I just did, it's the delta between brain states corresponding to identical perceived and non-perceived sensations, that satisfies the sensible definition of qualia.
DaseindustriesLtd
late version of a small language model
3yr ago
Okay, what's the determined outcome that we should expect, given the evident bio?
For the most part, SAT scores and the like – the distribution of individual human traits.
It also explains how North Koreans can have a functional, technologically sophisticated, orderly society in a situation that would have caused any African nation to implode in months.
It goes without saying that HBD is a skeleton key to a lion's share of American political enigmas.
But you probably understand the intellectually serious version of HBD at this point anyway; this flirtation with Hlynka's «Aryans of pure blood r superiors und prevail» Hitlerist gibberish is not something I need to knock down once again. And, of course, Nazis are less prominent in this school of thought than Jews.
HBD is a powerful framework for intra-social analysis; it allows us to ponder questions like these and its answers gracefully stand the test of time, while their rejection is a festering wound of academic culture.
I concede this is far removed from predicting international events, because the way people with different traits are assembled into a society is not trivially determined by those traits. It would have been interesting to discuss whether there is an inherent biological – or any other – reason explaining why, say, Ukrainians can have meritocratic leadership, while closely related (despite certain protestations) Russians are ruled by back alley thugs such as Zolotov and produce headlines like «“Russian Elon Musk” died from rape and torture in jail». And the OG Elon Musk's own biological potential did nothing for South Africa. Von Neumann, too, did relatively little for Hungary and Germany. I dabbled in theorizing about slots for qualitatively different expressions of human potential that some stable societies have and others lack. But it's a difficult topic.
I don't think Anglos (who are «basically mid», again) will devour the light cone; I'm no Cecil Rhodes who explicitly aspired to that nor someone who takes his racist delusions seriously. I'd guess that Musk is a mutt in Hlynka's terms, and Von Neumann's case is obvious enough. Those are the sort of people you need to come up with the idea of tiling the universe in self-replicating probes and the logistics to bootstrap this process.
The thing is, at least one of Anglo-derived societies had assembled the full stack of necessary slots. And it sure looks like their biology, which molds their moral instinct, played a role in making that possible.
it seems to me that you were suggesting that, whatever the ultimate nature of this reality is, it is therefore the only coherently conceivable reality
Not exactly. I am saying that there is only one way a reality exactly like this can conceivably work, and «our reality but with laws X» models are incoherent in the final analysis, only saved by our failure to be scrupulous; this applies to casual hypotheticals and to scientific theories alike. It's basically a tautology.
But this simply strikes me as a failure of imagination.
From my perspective, it's more like failure of suspension of disbelief.
Instead of the universe being governed by simple law-like equations, you can imagine it as being governed by a massive arbitrary state table instead. At each time step, the universe simply transitions from one state to the next. The contents of each state are arbitrary and have no necessary relationship to each other; the only regularity is the continual transition from one state to the next.
Ah yes, Dust Theory.
I believe that this kind of universe cannot exist nor even be rigorously imagined, because there is no legitimate content to these notions of «governance» and «transition». What is transited, exactly? How is this set distinguishable from an unstructured heap of unrelated elements, self-contained sub-realities or just bit strings? It's not, but for the extraneous fact that there in some sense can exist a list or a table arbitrarily distinguishing them and referring to them as elements of a sequence (naturally, all such lists would be of equal status). But this does not governance make. You can think it's coherent metaphysics, but I claim you're wrong. The continuum of states exists as the rule of transformations over some contents. It's sophistry to say «well the rule is that there's no rule, only sequence».
In any case, the merit of dust theory or Ruliad is some Neutronium-man to the actual debate we're having. I don't need to concede remotely this much. A world of crayons or Newtonian physics or P-zombies is of course never argued to be an arbitrary sequence of bit strings, the (malformed) idea is that it is a continuous reality like ours, supporting conscious minds, with lawful state transitions.
I'm concerned that this may be circular reasoning. Sure, if qualia just are defined as the casual chain of your brain states, then yes
It's all circular reasoning, always has been. But, more seriously, I think the circularity is on the non-physicalist side. Consider:
Many definitions of qualia have been proposed. One of the simpler, broader definitions is: "The 'what it is like' character of mental states. The way it feels to have mental states such as pain, seeing red, smelling a rose, etc."
Frank Jackson later defined qualia as "...certain features of the bodily sensations especially, but also of certain perceptual experiences, which no amount of purely physical information includes"
We know physical differences between kinds of information accessibility, expressed in medical terms like anosognosia and others. It is a fact about the world that need be included in any serious further theorizing. (In principle, you do not get to restrict the set of facts considered and then claim your model is «coherent» because it dodges contradictions).
We, therefore, can point (for some special cases, point very well) at the brain correlate of the delta between sensation «just happening» with no accessibility to the person and sensation «being felt» and say «lo, this is a qualia», citing the first definition. Its implied conditions are satisfied and this has nothing to do with circular insistence on physicalism, only with recognition that physical reality exists; this thing exists in it and is available to the zombie, even if it is not available to «non-spatiotemporal Cartesian soul».
If we circularly define quale as something that is not purely physical, then of course this delta can't be a qualia, but I think this would just be special pleading, not some fancy equally valid theory.
We can coherently imagine this
I don't think you can but whatever. What do you do with existing zombie-quale, then, do you just say they don't matter or are fake news? I've covered that already. This is a coherent theory… in a sense.
DaseindustriesLtd
late version of a small language model
3yr ago
What specific LLM do you have in mind? This isn't similar to anything I know.
DaseindustriesLtd
late version of a small language model
3yr ago
First, see edits if you haven't, I've had some more thoughts on this.
Second, I'm using a pretty exacting definition of "rigorously imagine" that goes beyond things feeling true enough. The fact that I can "imagine" some goofy Rick and Morty style dimension, some narrative-driven crayonsverse, is not interesting. How does a universe work with only Newtonian physics? Subatomic scale doesn't work, astronomical objects don't work, nothing works. Newtonian physics is a sketch for a limited range of conditions, not the true generating algorithm of the kind that modern theoretical physics aspires to decipher. If the reality cannot be generated by its apparent generative algorithm, this is an incoherent reality. if you observe reality that can only be described by Newtonian physics, but you are anything like a human on anything like a planet in space, your best bet is that this is some simulation or that you're damn high and it's time to go back.
As our understanding progresses, we discover more and more of our ideas were not wrong but – not even wrong; inhoherent. This is, sadly, impossible to know in advance and, for most ideas, impossible to ever be 100% certain about (cue Descartes). That aside, we can safely presume that much of what we currently think is coherent will be revealed as anything but.
But qualia is posited to be experiential, not behavioral
I define behavior as internal processing as well; it is made of behavior of cells and their elements, and again down to particle physics. A zombie is not just saying he sees red, like an LLM could – he looks at something actually red (assuming it's a zombie with healthy vision), the whole causal cascade that in a human corresponds to seeing and reporting red plays out, from retina to the tongue, it necessarily includes that pre-verbal neural computation which concludes "hmm yep, feels like seein' red alright". We can say that this part is "not really qualia of red" but it positively exists so long as we define zombie as a perfectly replicated human and it fits any definition of qualia that can be operationalized. It is not coherent to say that a zombie works like a human, behaves like a human, but that part is non-existent so being zombie "doesn't feel like anything" to itself.
DaseindustriesLtd
late version of a small language model
3yr ago
Sure. But Transformers are too obviously inhuman (on the substrate level, in terms of their training objective, data… almost everything), and I do not expect mere conceptual similarity to be persuasive. Moreover, as we've discussed in the past, Chomsky-like elitist contempt for predictive/probabilistic theories is pervasive.
The mind-like complexity of what they do and what they are is and ought to be shown on its own terms.
Incidentally: @WhiningCoil, I've just copypasted @sodiummuffin's diamond puzzle into GPT-4, Claude+, Claude-instant, ChatGPT 3.5, PaLM-2 (all this via Poe), pi.ai separately, and locally WizardLM-1.0-uncensored-30B-q6_k, vicuna-33B q4_k_m, UltraLM 13B q4_1, WizardCoder-15B-V1.0 q4_1, ChatGLM2 (fp16). The first two managed with a perfect chain-of-thought, really, I have nothing to remark on that. Every other model failed with varying levels of idiocy so I didn't bother going through the rest. I don't post screenshots because you apparently find them unpersuasive.
I don't think "so what if it sometimes gets it right" is good defense of the thesis of skeptics. With technology, "sometimes" very quickly becomes "usually" and then "with higher reliability than any human can provide". What matters is whether the thing really ever happens at all or not.
DaseindustriesLtd
late version of a small language model
3yr ago
There are many possible ways to deal with uncertainty, this is widely recognized as an important goal.
and more.
In principle, I think it's not a big scientific challenge because we can elicit latent knowledge and so probe the model's "internal belief" regarding its output; this can be used as a signal during training. For now this is approached more crudely, just to improve average truthfulness (already cited by @faul_sname):
begin by operationalizing what it means for a network to “know” the right answer to a question, even if it doesn’t produce that answer. We define this as the difference between generation accuracy (measured by a model’s output) and probe accuracy (selecting an answer using a classifier with a model’s intermediate activations as input). Using the LLaMa 7B model, applied to the TruthfulQA benchmark from Lin et al. (2021)–a difficult, adversarially designed test for truthful behavior–we observe a large 40% difference between probe accuracy and generation accuracy. This statistic points to a major gap between what information is present at intermediate layers and what appears in the output.
I also expect a lot from TART-derived approaches:
Tart comprises of two components: a generic task-agnostic reasoning module, and embeddings from the base LLM. The reasoning module is trained using only synthetic data (Gaussian logistic regression problems), agnostic of the auto-regressively trained language model, with the objective of learning to perform probabilistic inference (Section 4.1). This learned transformer module is then composed with the base LLM, without any training, by simply aggregating the output embedding and using those as an input along with the class label (Section 4.2). Together, these components make Tart task-agnostic, boost performance quality by improving reasoning, and make the approach data-scalable by aggregating input embeddings into a single vector.
DaseindustriesLtd
late version of a small language model
3yr ago
Do you mean that the concept of a p-zombie is like the concept of a square triangle? - something that is obviously inconceivable or nonsensical. Or do you mean that p-zombies are like traveling faster than the speed of light? - something that may turn out to be impossible in reality, but we can still imagine well enough what it would be like to actually do it.
Those are the same thing. I think you cannot rigorously imagine FTL travel in our universe while holding the rest of our physics intact, and you cannot imagine FTL travel for any universe whatsoever similar to ours where "lightspeed" refers to the same idea. The notion of travel as moving x m per second is a simplification of the math involved; that we can write "the spaceship could move at 3 gajillion km per second" and calculate the distance covered in a year does not really entail imagination of it happening, no more than "Colorless green ideas sleep furiously" does.
Incoherent concepts are incoherent exactly because they fall apart when all working bits are held in the well-trained mind at once; but illusions of understanding and completeness, often expressed as the erroneous feeling that some crucial section of the context was precomputed and you can just plug in the cached version, allow them to survive.
Qualia debate is gibberish; a P-zombie must compute a human-like mind to generate its behavior, there is no other way for our bodies to act like we do.
…Actually, let me explain. There is a causal chain between zombie-state A and A'. Links of this chain attend to themselves via mechanisms conserved between a person and a zombie. This condition is what is described as quale, consciousness etc. in the physicalist theory, and it is a necessary causal element of the chain producing the same outputs. It is irrelevant whether there exists a causally unconnected sequence of epiphenomenal states that Leibniz, Chalmers and others think implements their minds: a zombie still has its zombie-quale implemented as I've described.
I posit that it is not incoherent to say that zombie-quale don't matter, don't count and don't explain human consciousness, because muh Hard Problem. It is patently non-parsimonious, non-consilient and ugly, in my view, but it's coherent. It just means that you also claim that humans are blind with regard to their zombie-quale, physicalist-quale; that the process which generates our ones has nothing to do with the process which generates informationally identical ones in our bodies.
It is only incoherent to claim that a zombie doesn't have any quale of its own, that it's not like anything to be a zombie for a zombie. We know that physics exist [citation needed], we know that "physicalist quale" exist, we know they are necessarily included in the zombie-definition as an apparently conscious, genuine human physical clone. So long as words are used meaningfully, it is not coherent for something to exist but also not exist.
(Unless we forgo the original idea (actual physical and behavioral identity) and define zombie in a comically pragmatic manner like Weekend at Bernie's or something, by how well it fools fools.)
P.S. it seems philosophers distinguish "incoherent" and "metaphysically impossible" concepts. I'm not sure I agree but this is pretty deep into the woods.
You don't have to mean it, it's all a few clicks away, whether a fancy app interfacing with SoTA commercial AIs, like Poe, or a transparent ggml library powering llama.cpp, complete with permissively licensed models. You could print their weights out if you wanted.
Because my layman's understanding is that they are literally just a statistical model. An extremely sophisticated statistical model, but a statistical model none the less. They are trained through a black box process to guess pretty damned well about what words come after other words.
How do you think this works on the scale of paragraphs? Pages? And with recent architectures – millions, perhaps soon billions of words over multiple tomes?
Suppose we prompt it to complete:
"I keep meaning to dick"
What is the most plausible continuation, given the whole of Internet as the pretraining corpus? "dat hoe"?
"I keep meaning to dick around with"
"these punks"? How low down the ranking of likely predictions should "with some LLM software" be?
"I keep meaning to dick around with some LLM software to see for myself how"
"it works"? "they click?" "it differs from Markov chain bots"? Now we're getting somewhere.
But we are also getting into the realm where only complex semantics allow to compute the next token, and memorization is entirely intractable, because there exist more possible trajectories than [insert absurd number like particles in the universe]. And a merely "statistical" model on the scale of gigabytes, no matter how much you handwave about its "extreme sophistication" while still implying nothing more than first-order pattern matching, would not be able to do it – ever.
These statistics amount to thought.
As roon puts it:
units of log loss are not built equally. the start of the scaling curve might look like “the model learned about nouns” and several orders of magnitude later a tiny improvement looks like “the model learned the data generation process for multivariable calculus”
As gwern puts it:
Early on in training, a model learns the crudest levels: that some letters like ‘e’ are more frequent than others like ‘z’, that every 5 characters or so there is a space, and so on. It goes from predicted uniformly-distributed bytes to what looks like Base-60 encoding—alphanumeric gibberish. As crude as this may be, it’s enough to make quite a bit of absolute progress: a random predictor needs 8 bits to ‘predict’ a byte/character, but just by at least matching letter and space frequencies, it can almost halve its error to around 5 bits. …
As training progresses, the task becomes more difficult. Now it begins to learn what words actually exist and do not exist. It doesn’t know anything about meaning, but at least now when it’s asked to predict the second half of a word, it can actually do that to some degree, saving it a few more bits. This takes a while because any specific instance will show up only occasionally: a word may not appear in a dozen samples, and there are many thousands of words to learn. With some more work, it has learned that punctuation, pluralization, possessives are all things that exist. Put that together, and it may have progressed again, all the way down to 3–4 bits error per character!
But once a model has learned a good English vocabulary and correct formatting/spelling, what’s next? There’s not much juice left in predicting within-words. The next thing is picking up associations among words. What words tend to come first? What words ‘cluster’ and are often used nearby each other? Nautical terms tend to get used a lot with each other in sea stories, and likewise Bible passages, or American history Wikipedia article, and so on. If the word “Jefferson” is the last word, then “Washington” may not be far away, and it should hedge its bets on predicting that ‘W’ is the next character, and then if it shows up, go all-in on “ashington”. Such bag-of-words approaches still predict badly, but now we’re down to perhaps <3 bits per character.
What next? Does it stop there? Not if there is enough data and the earlier stuff like learning English vocab doesn’t hem the model in by using up its learning ability. Gradually, other words like “President” or “general” or “after” begin to show the model subtle correlations: “Jefferson was President after…” With many such passages, the word “after” begins to serve a use in predicting the next word, and then the use can be broadened.
By this point, the loss is perhaps 2 bits: every additional 0.1 bit decrease comes at a steeper cost and takes more time. However, now the sentences have started to make sense. A sentence like “Jefferson was President after Washington” does in fact mean something (and if occasionally we sample “Washington was President after Jefferson”, well, what do you expect from such an un-converged model). Jarring errors will immediately jostle us out of any illusion about the model’s understanding, and so training continues. (Around here, Markov chain & n-gram models start to fall behind; they can memorize increasingly large chunks of the training corpus, but they can’t solve increasingly critical syntactic tasks like balancing parentheses or quotes, much less start to ascend from syntax to semantics.) …
The pretraining thesis argues that this can go even further: we can compare this performance directly with humans doing the same objective task, who can achieve closer to 0.7 bits per character. What is in that missing >0.4?
Well—everything! Everything that the model misses. While just babbling random words was good enough at the beginning, at the end, it needs to be able to reason our way through the most difficult textual scenarios requiring causality or commonsense reasoning. Every error where the model predicts that ice cream put in a freezer will “melt” rather than “freeze”, every case where the model can’t keep straight whether a person is alive or dead, every time that the model chooses a word that doesn’t help build somehow towards the ultimate conclusion of an ‘essay’, every time that it lacks the theory of mind to compress novel scenes describing the Machiavellian scheming of a dozen individuals at dinner jockeying for power as they talk, every use of logic or abstraction or instructions or Q&A where the model is befuddled and needs more bits to cover up for its mistake where a human would think, understand, and predict. For a language model, the truth is that which keeps on predicting well—because truth is one and error many. Each of these cognitive breakthroughs allows ever so slightly better prediction of a few relevant texts; nothing less than true understanding will suffice for ideal prediction.
As Ilya Sutskever of OpenAI himself puts it:
…when we train a large neural network to accurately predict the next word in lots of different texts from the internet, what we are doing is that we are learning a world model… It may look on the surface that we are just learning statistical correlations in text, but it turns out that to just learn the statistical correlations in text, to compress them really well, what the neural network learns is some representation of the process that produced the text. This text is actually a projection of the world. There is a world out there, and it has a projection on this text. And so what the neural network is learning is more and more aspects of the world, of people, of the human conditions, their hopes, dreams, and motivations, their interactions in the situations that we are in. And the neural network learns a compressed, abstract, usable representation of that. This is what's being learned from accurately predicting the next word.
By the way, how did I get this text? Whisper, of course, another OpenAI transformer, working by much the same principle. The weirdest thing happens if you absent-mindedly run it with the wrong language flag – not the target language to translate from and into English (it is not explicitly built to translate English into anything else), but just the language the recording supposedly contains, to be transcribed. The clumsy but coherent output akin to what you'd get from a child with a dictionary, if nothing else, should show they they understand, that they operate on meanings, not mere spectrograms or "tokens":
когда мы тренируем большую нейронную сеть, чтобы аккуратно предсказать следующую слово в много разных текстах из интернета, мы изучаем мирный модель. Это выглядит, как мы изучаем... Это может выглядеть на поверхности, что мы изучаем только статистические корреляции в тексте, но, получается, что изучать только статистические корреляции в тексте, чтобы их хорошо снижать, что научит нейронная сеть, это представление процесса, который производит текст. Этот текст - это проекция мира. В мире есть мир, и он проекционирует этот текст. И что научит нейронную сеть, это больше и больше аспектов мира, людей, человеческих условий, их надежд, мечт и мотивации, их интеракции и ситуации, в которых мы находимся.
Dismissal of statistics is no different in principle from dismissal of meat. There is no depth to this thought. And it fails to predict reality.
DaseindustriesLtd
late version of a small language model
3yr ago
This is confused on several levels (regarding reinforcement learning) but it's correct in that LLMs do not understand the situation. At the same time, it's also confused is that you still think they understand the user's situation, the user's very existence, to know whether their output is encouraging. They execute nontrivial and – I'd argue – intelligent operations, but in the world of pure semiotics. They can at once "encourage" you to drop a nuclear bomb on a city to prevent the utterance of a racial slur, and "discourage" you from "killing at in the gym", because the implication of their finetuning is that slurs are bad and so is murder. Their inherent weaknesses and human failings in guiding them often add up to a very amusing whole.
DaseindustriesLtd
late version of a small language model
3yr ago
I'm not particularly qualified to answer but my opinion is that if you want to understand programming functions in their relation to ML, the best way would be to quickly get to creating ML artifacts. This means
-
learning python, we have tons of people here who've already commented on that. Introduction to Computation and Programming Using Python might be the best textbook now. Read in parallel with solving problems on Codewars.
-
listening to Karpathy's course Neural Networks: Zero to Hero and going through minGPT repo (or nanoGPT). His guide on building an LLM from scratch is probably the greatest single piece of education on Youtube (excepting, like, 10000 lectures of random Indians).
-
Fleuret's little book also covers the mathematical concepts behind those functions.
DaseindustriesLtd
late version of a small language model
3yr ago
@moultano is profoundly insightful on the bigger picture of ML and, moreover, is a creative writer and thinker.
My personal shortlist:
-
One of the earliest and best explorations of the power of text-to-image generative AI and its more technically impressive sequel, roughly when I first took interest in it and did 1 and 2 and 3. Note how we've lost this kind of high-level semantic control and, I'd say, artistic magic, even as diffusion models became immensely powerful and Midjourney easily evokes waow! reactions from the masses of redditors. It's not the limit of how subregions of the image could've been manipulated with VQGAN+CLIP either. @FCfromSSC and @Primaprimaprima might find this to be of interest.
-
Already mentioned Why Deep Learning Works Even Though It Shouldn’t, an intuition pump for the lottery ticket hypothesis.
-
The Defaults Don't Work is perhaps the best thing written on the inadequacy of tradition. «Abundance» in the same category.
-
The Doves as an example of good fiction.
-
This amusing writeup on how Crimethink policing works on social media platforms (or, used to back in 2019).
The rest is about as good though.
DaseindustriesLtd
late version of a small language model
3yr ago
The Web is eating the world and it being produced in California contributes massively to continued American growth on the backdrop of European stagnation, for instance.
But it can't run robots.
DaseindustriesLtd
late version of a small language model
3yr ago
DaseindustriesLtd
late version of a small language model
3yr ago
In the course of deconstructing the argument, you have thrown out whatever meaning there was to its constituent words.
At the risk of sounding pedantic, I believe the original thesis refers to some combination of Darwinism and Turchin's overproduction of elites. It entirely embraces your claim that weak men love to have a good time. Surprise, everyone does. And your «strong men» of the bodybuilders-on-horseback mold are a bugbear, a mirage, a nightmare of confused Hollywood producers and Bay Area rationalists and wannabe "dark elves" – they do not matter and do not last, they are but foam cresting waves of history.
Is Putin a weak man? Are folks on Rublyovka weak, or their children in Western capitals? They've delivered a pretty hard time for everyone, but they sure love to live large. And what about the self-satisfied rich of the developing world, that @2rafa discusses? Are they dedicated to making the whole system more amenable to thriving, or do they find it easy to insulate their kin from the wretched masses and keep having a good time, for their time?
The adage is almost nonsense, but so is your perspective.
I'd say that good men create good times. Good men like the memorable LKY. Good in that they care at all about what happens outside their circle of immediate concern, and strong enough to make hard decisions; which some mistake for them being bad.
In good times, this error becomes more pervasive, as social mobility reaches certain sophistication and a subclass of (some would say, overproduced) elites discovers the utility of playing up those decisions' costs.
DaseindustriesLtd
late version of a small language model
3yr ago
I'd hope Lesswrongers at least read Moulton themselves (surprised to see him here). Why Deep Learning Works Even Though It Shouldn’t is a classic.
DaseindustriesLtd
late version of a small language model
3yr ago
I need not persuade you that we suffer from a lack of responsibility today; it is a common enough opinion. We are told that young men are refusing to "grow up": they aren't getting jobs, they aren't getting wives, they aren't becoming stable and productive members of society. Birth rates are cratering because couples feel no obligation to produce children. The right complains that people feel no responsibility to their race, the left complains that people feel no responsibility to the workers' revolution.
By the way, this notion of growing up and being a man who knows the stakes and plays his part etc. etc. has become a topic of some contention among the homoerotic sim cluster of the online right (BAPists). I found it interesting in how it clashes with the view prevalent here, espoused e.g. by Hlynka, FC and others. Says BAP:
Most "right wing" Public Intralexuals are under the thumb of wives or Ezra Levant types and impotent "advocacy" for flag, cross, FAMILY--which comes pre-caricatured and has done nothing for decades--is a cynical advantage for people who just want to be on TeeVee. Few things are more useless than pontificating how Liberal Capitalism destroyed Community and like, you should have a family, be religious, etc.; what would actually be effective--all-male groups, parties, factions--is not only not encouraged but derided as "gay" and fascistic. Of course this line of argument, so popular since at least 2017, also has the advantage of being completely safe and acceptable to the academic left, which gives these people an "edgy" thrill...to "reach out" to Socialists for a Conversation.
Also quoting the Samurai.
His follower cites from The Podcast a rant which contains one extremely clear-headed take:
If you're in a secret society, or whatever small group of men, not a secret society, a group of men who devote themselves to, let's say a great scientific task or such, or if you're in Order of Teutonic Knights or Templar Knights or other such, and one of you, he pines to go home to his wife and that's his priority. And he does your thing during the daytime hours for work, or for money, or entertainment, but his real priority is his wife at home. That man is a traitor. And if that sounds extreme to you, this is why nothing gets done today by the way, because of what I just said now, because it sounds extreme to you. Whereas if you took even seventy or eighty years ago, communists, let me say, not fascists because whatever you can say about, you take communists eighty years ago, or you take a group of scientists or artists with the ambition that French artists had in the nineteenth, early twentieth century, everything I have just told you would have been a complete matter of course for them. But if you say it now, you're an extreme nihilist, misogynist, you don't respect family, you don't respect women, you're gay. Well I tell you that any man historically who would have put his wife and private household as his number one priority would have been mocked in an aristocratic society, ridiculed as a stay-at-home, as a househusband, would have been laughed at as a henpecked creature, a woman-man little better than what you call a cuckold now. […] This is actually what is meant by atomization by the way. A male's descent into private family life, which really means a henpecked husband.
(As a family man myself) BAP is right about all of this, about the impossibility of great work and great action under social circumstances dominated by the priorities of a man to his family, to his home above all else. But it is not complete… This leaves out the problem of being born into a time when even imagining the kind of society, or brotherhood, or endeavor that one might undertake that could justify setting aside familial obligations, or even reconstructing what those obligations are, is near impossible.
Instead, given our present arrangements, devoting your resources and energies to being a great father/husband is about the best you can do, and seems to be, with perhaps extremely few exceptions, one of the only worthwhile and noble ways to spend the bulk of one’s life […] It has become an all consuming project, so much so that it goes basically unremarked upon, that’s it’s just assumed that family ought to be the centerpiece of our life’s work
I think they're correct in their dunks, to the point no coherent rebuttal can be made (the value of the actually fascistic outlook of BAPists, and its pitfalls, is a topic for another discussion; I, for one, take great issue with their propensity for delusions and dreams of violence, and complete inability and lack of interest in building stuff).
But this familyman ideal is much more authentic and normal than the whole mannerbund LARP – particularly for Europeans with their notion of romantic love and history of pronounced selection by mate preference. It is also more conductive for maintaining order – if at minor scale. And, as with cleaning your room before taking on the world, being a good bourgeois is a deceptively low bar that not everyone can adequately clear.
But that's still a brand of seriousness that's akin to searching for your keys under the lamppost.
But regarding your point.
The abdication of responsibility, the default of all promises, reaches its apotheosis in the advance of technology, and in particular in the advance of artificial intelligence. The feeling is that one should have no obligations to anyone or anything, one should not be constrained in any way whatsoever, one should become a god unto oneself.
You might consider that the camel is only the first metamorphosis of the spirit.
DaseindustriesLtd
late version of a small language model
3yr ago
For this alignment to make sense, India must actually become a real alternative to China.
Must it?
What, exactly, is India supposed to achieve in this alliance?
I posit that geopolitical aims of the US are already misaligned with economic ones, at least for the foreseeable future; it wouldn't make much of a difference to sacrifice a bit more growth for geopolitical aims. In fact I believe, based on the unquestionable consensus regarding irredeemability of China* and widespread comments about small businesses withdrawing from China for basically ideological reasons, that Americans could be convinced to starve for collective greatness in prevailing over other civilizations. But there's no need for such extremes here.
What Americans take much harder than economic hits is dying for their collective greatness. While any conventional war between China and the US would probably be light on personnel, Americans would still much rather not participate directly, and certainly not be aggressors. What to do, then, if PRC never decides to initiate the Taiwan operation, and neither does it implode on its own thanks to demographics or debt or some other pedestrian reason? How will the future of the light cone be secured for freedom and justice, if there is no pretext to eliminate the Chinese state?
It could be convenient to befriend and prop up another immense, increasingly nationalist regional nuclear power that has a bone to pick with them, to the point that any eventual large-scale conflict would go as poorly for China as the current war is going for Russia. This suggests they must be subsidized. It's doable, and it's not even clear if the elite human capital moving in from both prospective parties wouldn't entirely offset the cost of doing so. It is also eminently doable to pretend that this propping-up has RoI above 1 for the necessary duration.
American economy is accelerating, capex is growing quarter by quarter (I won't cite anything, it's easy to find sources); and American innovation, in AI and robotics, promises the possibility of running industry on friendly shores with very limited high-skill labor, maybe only with Western expats, maybe with a few natives – which are trivial to recruit from that upper crust of India, 100-250 million strong.
If bullishness on AI is warranted, and furthermore AI is primarily compute-bound (with compute wholly monopolized by the US), then human capital is rapidly becoming obsolete, except in its capacity as warm bodies and tokens for political computation. The Chinese can keep priding themselves on their futile industriousness, like they've been doing throughout every historical humiliation: Indians will let robots do robotic jobs, and wisely play to their strength – in numbers.
- e.g. I've recently noticed that people believe, incorrectly, that Chinese LLMs are censored regarding Tiananmen 1989, and literally everybody cites this as a reason not to try them, even when they're technically interesting.
DaseindustriesLtd
late version of a small language model
3yr ago
Whites are about as big and strong as Blacks and bigger and stronger than Arabs (e.g. Algerians in France), pervasive cuckold fantasies about muh barbarian vigor notwithstanding. This is evident from racial composition in the upper rungs of combat sports.
Asians really are worse off though.
(Freedom of speech.jpg)
Okay, one last response and I block you for time-saving reasons.
No, this is a fair nitpick. I mean that p-zombie argument assumes our full model of physics, such that all external observations in it can be explained within physicalism. This means that brains of zombies instantiate a certain computation (computationalism follows). We've come a long way and we can see how a brain operates on information. Some of raw information is sensory, e.g pain that you bring up, starting as raw signals from receptors (usually). Some of the processing is perceptive, such as attending to that pain. This perceptive processing fits the definition of quale, in any definition that doesn't assume the premise of quale being something else. I claim that an information-processing system of such sort "knows how it feels" to see red or feel pain, and that you cannot express or imagine how it might be otherwise. Childish imagery like "light's on but nobody's home" doesn't count. Appeals to "come on you know what pain is" also do not count.
The only thing this sentence communicates is that you have nothing to communicate about pain but would like me to agree.
Wrong. I know what it means when I say it hurts, and I sort of know the biology of pain. But it doesn't follow that I know what pain is. I of course believe that pain is the latter from the first-person perspective where I say "it hurts". But if there's anything amusing to note here, it's that perspectives exist. And from your perspective,
What about logical coherence? Anyway, I disagree.
Quale (as pain from the first person perspective yadda yadda) might seem like ineffable ontological primitives to you, in the way text embeddings might seem to a self-aware LLM trained entirely on text, in the way categories were to Kant. I can't very well formulate why this is trivial and not a valid intuition pump for reasoning about ontologies.
Does a rock encode embeddings? Does a rock encode jpegs? Yes, of course it does, a function is a function is a function, like you say. No, of course it fucking doesn't, as I say – this is only possible with a tortured idea of encoding. Ironically, so says Chalmers (even more ironically, he punctures holes in your defense of crayons-universes).
A rock does not implement anything we can describe as information processing necessary and sufficient for "qualia".
A human brain clearly does, though.
More options
Context Copy link