site banner
PaperclipPerfector  10mo ago (text post)   34932 thread views

Culture War Roundup for the week of June 26, 2023

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

11
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

VisionOS and the Future of Input


Ever since the computer first arrived, keyboard and mouse has been the standard. You have a flat surface with raised little squares that you smack with your fingers. You have another little rounded shape with a flat bottom you move around, and click with.

This awkward, clunky interface has significant culture war elements, in that an entire class of powerful people arose - specifically people who didn't have traditional status markers like height, strength, or indomitable physical presence. Instead these 'nerds' or 'geeks' or whatever you want to call them specialized themselves in the digital realm. Now, the Zuckerburgs and Musks of the prior generation rule the world. Or if they don't, they soon will.

These outdated interfaces seem perfectly normal to everyone who has only used them. Sure many people have used a controller for video games, and may think that controllers are superior for some cases, but not others. Keyboard and mouse is the only way to operate when it comes to a computer, most people surely imagine.

That being said, it's actually quite easy to dip your toes into alternate input methods. Talon is a system that utilizes voice to let you do practically anything on a computer. You can move the mouse, click on any object on your screen, dictate, edit text, and you can even code quite well. Talon's system even supports mapping operations, sometimes very complex ones, to custom noises you record on your own.

On top of that you can integrate eye tracking, using a relatively inexpensive device. If you've ever used voice control combined with eye tracking, you can operate around as fast as someone who is decent at using a keyboard and mouse.

If you have ever used these systems, you probably know that because most digital setups are built for keyboard and mouse, it's not necessarily perfect. Keyboard and mouse still hold the crown.

But. There is a certain magic to controlling a computer through your voice, or your eyes. It begins to open your mind to new possibilities, the idea that there are better, faster, easier, more natural ways of interfacing with a computer than the defaults we have been stuck with.


Enter Apple's VisionOS.

If you haven't seen the recent demo of Apple's new VisionOS they're breaking brand new ground. The entire OS is built around looking at things, and making minute hand motions to control the icons you're looking at. There are no controllers, no physical interfaces whatsoever besides your eyes and your hands. It's breathtaking to watch.

In a review from John Gruber, a well respected old head in the VR space and a creator of markdown, the possibilities behind this new technology are apparent. Gruber describes how

First: the overall technology is extraordinary, and far better than I expected. And like my friend and Dithering co-host Ben Thompson, my expectations were high. Apple exceeded them. Vision Pro and VisionOS feel like they’ve been pulled forward in time from the future. I haven’t had that feeling about a new product since the original iPhone in 2007. There are several aspects of the experience that felt impossible.

Now Apple does tend to get a ton of hype, but this reaction of being amazed by the experience is surprisingly common among earlier reviewers:

Similarly, Apple’s ability to do mixed reality is seriously impressive. At one point in a full VR Avatar demo I raised my hands to gesture at something, and the headset automatically detected my hands and overlaid them on the screen, then noticed I was talking to someone and had them appear as well. Reader, I gasped.

The implications of this 'spatial operating system' are varied and multitudinous, of course. There will be all sorts of productivity gains, and new ways of interacting with the digital world, and fun new apps. However I'm most interested in how this innovation could shift the balance of power back to the strong and physically capable, away from the nerds.

No longer will clunky interfaces make sense - instead computers will be optimized around healthy, fully functional humans. Ideally the most intuitive and common control schemes will reward physical fitness and coordination. Traits which nerds lack in droves.

Will we see a reversal of the popularity that being a nerd or geek has gained in the past few decades? Only time will tell.

This strikes me as yet more technology for the "smart home enthusiast" type of person.

I look down on such people who fawn over their nonsense gadgets and have the gall to call themselves "tech people". All that rubbish does is introduce many more points of failure (both human, in the case of Amazon destroying a guy's smart home for an imagined slur, and technological) for extremely minor or questionable fringe benefits. Great, I can pre-heat my oven remotely on my way back from work, but it can also sometimes also decide to blaze to 400 degrees F in the middle of the night on its own.

Technophiles have all the latest privacy invading smart home rubbish that will brick itself within a year when the company either goes bust or decides to start charging you for features that previously were standard.

People who actually understand and work in technology have exactly one mid-2000s printer.

Technology is like karate; you ideally want to use it as little as possible. Over-engineered smartrubbish either breaks, bricks or takes 2 hours on the phone to tech support to make work in the first place... before it breaks or bricks anyway.

Technology is like karate; you ideally want to use it as little as possible.

I like this take. That being said, without the crazy technophiles how would boundaries be broken? How would we advance the state of the art, to let people like you get even more out of your tiny piece of technology?

See also: Hypebeasts as Honeybees, a post exploring the topic of bleeding-edge adoption, albeit from the firearms-and-accessories perspective.

I suppose in that sense they do serve a purpose.

No longer will clunky interfaces make sense - instead computers will be optimized around healthy, fully functional humans. Ideally the most intuitive and common control schemes will reward physical fitness and coordination. Traits which nerds lack in droves.

Have you ever watched a high-level FPS tournament? I'm a weakling but I swear I can type circles around most bodybuilders and sportsmen, much less normies. "We" are going to have significant advantages if the system rewards precision and reflexes.

Unless we integrate force-feedback exoskeletons into this visionOS, the death of the nerd's advantage is not yet neigh.

I’m willing to bet that precision and reflexes are heavily correlated with grip strength.

I think there might be a major confounder there. Drop that one and I'd guess the correlation is far less clear.

To clarify, is the confounder sex differences in reflexes and grip strength?

EG: the best esports players are almost all AMAB.

Yes.

I'm not saying it won't augment the current toolset in new ways I can't well predict. But, the nice thing about mice and keyboards is that, when I want the computer to respond, I touch them. When I'm not touching them, the computer isn't responding. I can do whatever I want with my hands and eyes, and they computer isn't trying to figure out whether I'm asking something of it. There's an inherent limitation with asking the entire attention of your eyes and hands or trying to constantly double task the input devices.

There an analogy with voice-to-text on phones. It may be easier or more convenient at times, but when you are also using your mouth to have an in-person conversation, it's worse.

Most of what you're saying is a nonsensical reach; a sci-fi user interface with gestures and blink-clicking doesn't favor anyone in particular. Mouse and keyboard wasn't invented to bypass jock physical superiority.

I've messed about with Windows Mixed Reality, doing basic computer tasks in VR, and it's kinda fun for the novelty, and allows for as many monitors as you want, but the idea that it will give jocks an advantage is (thanks for giving me an opportunity to use this word) Asinine.

Nerds have become common enough that they are no longer relegated to the fat out of shape basement dwellers of lore. Plenty of them are health obsessed or in shape. Zuckerberg is an example of this. I think if this interface change happens it will not impact their social status.

I also find it interesting that you claim eye tracking will benefit the in shape people. Do you know where this technology was originally developed? For helping paraplegics interface with computers.

I think this technology will find some traction. There has been a trend towards minimizing movement and effort to get more out of our computer interactions. However it's only going to be a temporary stop. The end goal is direct brain to computer interfaces. Why fire signals to tell your body to do something that tells a computer to do something? Why not just fire signals to tell the computer directly to do the thing? Elon Musk is trying to skip to that end goal with neuralink.

If I wanted to talk to my goddamn machinery, I'd have a microphone. I don't. I don't want to tell the computer "move here, do that, turn on this, type out the other thing". I can do that quicker with my fingers, and I can type and think at the same time, rather than having to stop and work out what I want the thing to do.

Eye movement control sounds even more hideous: so blink at the wrong time, and you just deleted all that hour's work.

fun new apps

KILL IT WITH FIRE is my immediate reaction. I don't want "fun new apps", I want the apps I'm already using to work for me, not for the company using them to data scrape every millisecond of my life.

If you haven't seen the recent demo of Apple's new VisionOS they're breaking brand new ground. The entire OS is built around looking at things, and making minute hand motions to control the icons you're looking at

And how do they pick up on the "minute hand motions"? I am always sceptical of demos because they are so curated and cherrypicked and then polished up later to produce the best looking experience (that may not be the correct one; how many hours of messing around with the headset did they edit down to that ten minute demo?)

If I'm wearing only a headset and "navigating with my eyes", how does the headset pick up that I'm pinching my fingers? I get the impression that you have to hold your hand in a particular way in a particular field of motion or else it won't work. I'm dubious. EDIT: Ah right, the thing is ringed with cameras that can see all. Now I'm even more dubious about what those cameras are looking at and recording and sending back to Apple when I'm wearing my snazzy headset so I can work on a spreadsheet without typing, just pinching my finger and thumb like crab claws.

Now if all you have to work with is your voice and/or eyes, sure, this is great, now you can use the computer. But I'm going to dinosaur on with my keyboard for now.

No longer will clunky interfaces make sense - instead computers will be optimized around healthy, fully functional humans.

Not necessarily, both reviews mentioned that the person wears glasses/uses contact lenses, so the headset was calibrated with their prescription (since you can't wear glasses and the headset at once). So they're already accommodating people with less than perfect eyesight, and if you have some mobility problem that you can't pinch your fingers, they'll probably invent something to adjust for that as well.

Eye movement control sounds even more hideous: so blink at the wrong time, and you just deleted all that hour's work.

Hah, this is so hidebound and backward I can't tell if you're being serious. I could just as easily say "click at the wrong time, you just deleted an hours work!" Get real.

KILL IT WITH FIRE is my immediate reaction. I don't want "fun new apps", I want the apps I'm already using to work for me, not for the company using them to data scrape every millisecond of my life.

Unfortunately technology doesn't sit stagnant for long, as much as you want it to. Sorry to break it to you friend.

And how do they pick up on the "minute hand motions"? I am always sceptical of demos because they are so curated and cherrypicked and then polished up later to produce the best looking experience

Not sure, but I linked two in-depth reviews from fairly popular writers that gush over it. There are far more people who have used it and find the control scheme incredibly intuitive and easy to pick up. That's with usually 30min - 1 hour demos.

  • -12

I could just as easily say "click at the wrong time, you just deleted an hours work!" Get real.

Bunky, I just deleted a long-ass comment elsewhere because I hit the wrong key by accident. I am not going to rely on "move your left pupil a nanometer to the left" as a control function and be confident "oh crap, I moved two nanometers and that is the delete!" won't happen.

They do gush over it, they also mention that the headset was personalised to them by a person measuring and fitting it and tutoring them in how to use it, that one of the control features was disabled, and that the entire experience took place in the setting provided by and set up by Apple.

How the kit works in the wild when it's your sitting room and you're trying to follow along to a Youtube tutorial is a different kettle of fish.

Hah, this is so hidebound and backward I can't tell if you're being serious. I could just as easily say "click at the wrong time, you just deleted an hours work!" Get real.

You aren't forced to twitch your finger regularly. Get real. Sorry to break it to you friend.

Hah, this is so hidebound and backward I can't tell if you're being serious. I could just as easily say "click at the wrong time, you just deleted an hours work!" Get real.

The difference is that in one case there's a bright line distinction between when you are and are not using the interface. Which becomes more pronounced in real world use vs tech demos. A gesture system that's comfortable when it's what you're focusing on can become much less so when you're casually using it and want to combine it with eating, drinking or just scratching your nose without accidentally triggering a gesture.

I could just as easily say "click at the wrong time, you just deleted an hours work!"

There are clear and obvious differences between "blink" and "click".

FWIW most vision based interfaces (including Apple's new headset) still use clicking precisely because of that reason, touch is much more intentional.

I sit on the couch. There's a glass of tea (yes, a glass) to the left; hopefully I won't hit it with my elbow and send into the stone floor again. Not wearing a blindfold certainly helps in this regard. I put laptop on, well, my lap, open The Motte, scroll to the end of your post, think a second, click «reply».

What, if anything, in all of this could have been improved by Vision Pro? Adding a dancing Mickey Mouse (partnership with Disney, wooo!) to the periphery? Fitting the website into a circular window superimposed on the room? Strapping the same laptop's motherboard to my forehead? Replacing touch typing with tiny finger gestures that are picked up by the IR sensor array under my nose?

Actually there are some ideas here. I expect great things to come of augmented reality. I envision a future of uncompromising transparency and sovereignty, with tastefully minimal HUDs and AI digital assistants that stay well out of the way while brutally suppressing incoming noise; athletic young people with 20/20 vision and perfect innocence about «dark patterns» who walk in the sunlight and look with concern and pity at hunchbacked millenials and zoomers squinting into their pocket surveillance devices. This can be done. Contra Strugatsky brothers, we don't need communism to get to see the brightest parts of the Noon.

But as roon convincingly argues, text is the universal interface and it is primarily the inherent power of text, not technical limits of the age, that decided the shape of The Mother of All Demos and the hardware paradigm that we're still living in. Why do you think large language models get almost no benefit from multimodality? Because nothing has more meaningful dimensions than a text string. Tablets and smartphones, this great civilizational achievement of Apple, offer a strictly lesser channel than text – beloved by people who'd never have a clue what to do with CLI. Now, I suppose there are designers and architects, and surgeons and such, and all those colorful applications from WWDC will truly shine. But… rotating a 3D model of a Zahi Hadid-esque building in a teleconference? Is this what the digital era is about? I guess PowerPoint will add some zany VR features soon and they'll be adored by the same type of person who inserted WordArt into business presentations in the 90s, but… really?

This reminds me again of that epiphany I had while watching Alita: Battle Angel, particularly the scene where Alita dodges an aesthetic chain attack (admittedly, under a certain influence that brings out visual elaboration): What waste! The CG artists could have gone so much wilder, added complex patterns of acceleration and inertia and homing; but viewers won't perceive such detail. We're long in the regime where our tools let us depict actions of posthumans, but our merely human brains make that power sterile.

It is a nontrivial undertaking to find a paradigm that in practice does better than an IDE, or CLI, or even the humble chat window – when you're limited by the user on the other side. Almost everything of worth that we do is text and ways to manipulate and chain and condition its blocks on different scales. Skeuomorphic gimmicks, graphs, trees, mindmaps, desks with sticky notes, kanbans – frankly, all either collapses into unwieldy mess while text keeps going, or is as close to vanilla text in spirit as to make no difference and not benefit from new peripherals whatsoever. Many have tried. Yet here we still are. When some of us will get Vision Pros and ability to render arbitrary shapes, they'll still be peering into a rectangular website with an input box and a button to send comment.

I hope people with better imaginations than mine will prove me wrong. I'm pretty fed up with our interfaces, as well as with the human condition in general. But gimmicks and fetishes, exciting and novel as they can be, are no more the answer than frivolous surgery. Another, genuinely superior way has to be found and explored.

Strapping the same laptop's motherboard to my forehead?

This would be amusing at least...

Replacing touch typing with tiny finger gestures that are picked up by the IR sensor array under my nose?

Yes, this is actually incredibly useful. For instance even with a limited interface like Talon, I will map certain phrases or words I use frequently in my job to a keyboard shortcut, or a noise. This mapping means that I save probably ~5 minutes of work per day. Over time if we can map more of these things to even more minute/simple actions, we are looking at serious efficiency gains.

When some of us will get Vision Pros and ability to render arbitrary shapes, they'll still be peering into a rectangular website with an input box and a button to send comment.

I disagree here, it may be a while coming but I do think we're in for a paradigm shift with regards to input.

Yes, this is actually incredibly useful. For instance even with a limited interface like Talon, I will map certain phrases or words I use frequently in my job to a keyboard shortcut, or a noise. This mapping means that I save probably ~5 minutes of work per day. Over time if we can map more of these things to even more minute/simple actions, we are looking at serious efficiency gains.

Not only is this something you can do right now on existing computers, it's much easier to do than with a noise/gesture system where the need for disambiguation makes custom definitions a much harder proposition.

Unless you're the sort of person who already has a bunch of autohotkey scripts for those tasks set up, you sure as hell aren't going to do that in a worse interface.

text is the universal interface

I have often assumed that you are the reincarnation of Erik Naggum:

The natural urge of people is to point, the mouse is the interaction tool that allows the user to do this.

do you actually believe this "natural urge" stuff? my "natural urge" (if I have any that exceed sex and hunger in complexity) is to speak or write. [...]

you live in your graphical user interface world with colors and idiots for users. I prefer people who can communicate and think in words. I have been to France once (Aix-en-Provence, a lovely little town, and Marseilles, a not so lovely city). I had to point at things because my French wasn't up to speed. I made an intense effort to learn to speak French, and after two weeks, I could speak it well enough to discuss the best packaging to send home a bunch of books I had bought in one of the excellent bookstores in Aix, after having spent a day in delighted (albeit patient) discussion with the bookstore owner. the thrill of being able to speak a new language was just exhilarating. unfortunately, I can't speak French, I can't even write French, anymore, but I can still read French, and I read German, as well. reduced to pointing, I feel like an illiterate moron. that feeling carries over into computing. pointing is for people who have yet to discover thought. in my view. you obviously disagree, but you won't see me agree to your "natural urge" bullshit any time sooner than you stop universalizing that bullshit to include me and millions of other people who feel disenfranchised by the now point-and-click "interaction" you want to make the universal mode of communication with a computer.

this has nothing to do with any Emacsen, anymore, but I just want you (and others) to know your "natural urge" nonsense is disputed by individuals who don't actually like your "interaction for dummies". and for those who are inclined to point and not think: this is an example of how one person (namely, I) think and have preferences, not a universalizable argument about what Emacs users prefer over what XEmacs users prefer. however, I'm still inclined to believe that XEmacs users (not the least after reviewing the comments I have received), are less verbal and more visual than Emacs users, are less interested in studying manuals and learning languages (case in point: XEmacs calls its Lisp an "API", Emacs calls its Lisp a programming language). David Hughes felt insulted by this, for God knows what reason. observation it is, and it has been confirmed by this debate, and in no way disputed.

a word says more than a thousand images.

exercises for the visually inclined: illustrate "appreciation", "humor", "software", "education", "inalienable rights", "elegance", "fact".

(Er, well the timeline doesn't match up at all for you to be his reincarnation, but you could be his brother or something.)

I'm not seeing this at all. First off, while the VisionOS sounds really cool, fundamentally it sounds pretty similar to the iPad - a device for consuming content, not creating it. Sure the interface is cool, movies look nice, and properly recorded immersive experiences are super awesome. But I don't see how it's a significant advantage in any type of content creation. For writing text and creating graphics, it's pretty much the same interface you can use now, except with a bigger virtual screen. You're going to be entering text by typing on a real or virtual keyboard I guess, or maybe speaking, either of which you can already do. If you want to do something more technical like writing code or system admin, it doesn't seem super well suited to the precision required. Sure it can do it, but it'll be emulating a conventional 2D UI with keyboard, so not really doing anything that you couldn't already do with any regular computer.

I guess you could also code by voice, per the video. But 1. I don't get that at all. Maybe you can type up some basic code if you know the special words and phrases, but it doesn't seem very practical to me to maintain or debug a large complex program. And 2. even if you could, you can already do that with a regular computer with a microphone and that software.

I don't see the supposed culture war impact at all either. First, I don't buy that "nerds" / tech workers are significantly less fit as a whole than the general population. I'm a developer and have worked with hundreds of others, they seem pretty average to me - a few are more muscular or athletic than usual, most have pretty standard office worker physique, a relative few look a bit more like the stereotypical nerd but are still perfectly capable physically. Next, even if that was the case, I don't see what VisionOS and related technologies would have to do with it - nothing about it is more physically intense than walking around a room. The number of people of all walks of life who would find anything about it the least bit physically challenging is probably effectively zero. Finally, as far as physical prowess being less important for societal status, that goes back at least to the start of the Industrial Revolution, if not further. Blame the backhoe, not the PC. A new user interface for PCs that may or may not make any difference in how they're effectively used won't make any difference at all.

I'm not seeing this at all. First off, while the VisionOS sounds really cool, fundamentally it sounds pretty similar to the iPad - a device for consuming content, not creating it

Do you not think people could dictate essays, draw with their hands, edit music, etc with this tool? Why does this entire OS seem fundamentally based on consumption to you?

Next, even if that was the case, I don't see what VisionOS and related technologies would have to do with it - nothing about it is more physically intense than walking around a room. The number of people of all walks of life who would find anything about it the least bit physically challenging is probably effectively zero.

It's not necessarily about physical intensity - it's about ease of use. Clearly I could expand on this point since many others seem confused as well. I'm betting that right now many people who would otherwise be more economically useful are not because they don't have the temperament, ability, or inclination to learn how to type quickly or move a mouse around quickly. With VisionOS and later generations, we'll see much more 'natural' inputs, or at least have a lower barrier to entry than, say, learning to type at 100 wpm.

Do you not think people could dictate essays, draw with their hands, edit music, etc with this tool? Why does this entire OS seem fundamentally based on consumption to you?

Sure, you could do that, but what's the advantage over conventional hardware? I can do all of those things just fine with any ordinary computer and off-the-shelf accessories and software that has been available for years. The only really new thing that this device has is the immersive VR experience. That's cool, but I don't see how that gets you anything for creating content.

I'm betting that right now many people who would otherwise be more economically useful are not because they don't have the temperament, ability, or inclination to learn how to type quickly or move a mouse around quickly. With VisionOS and later generations, we'll see much more 'natural' inputs, or at least have a lower barrier to entry than, say, learning to type at 100 wpm.

I don't think that's the case. Keyboards and mice have been dominant because they are very easy to use. Sure, it's not easy to type 100wpm with good accuracy, but pretty much everyone can type 10wpm. Typing 100wpm isn't that useful outside of stenography anyways. Typically the limiting factor is how fast you can think of more meaningful words of text or working lines of code to type, not how fast you can physically type them. In fact, I'd bet that whatever the solution VisionOS uses for typing (we haven't seen that yet, gee I wonder why that is, you'd think if they have an awesome solution to getting text from the user's mind to the device they'd have shown us), it's less intuitive than a keyboard. How much skill does it take to get 10 correct wpm into a VisionOS document versus a regular computer with a regular keyboard?

And that's before we get into things like, how easy is it to read text or data off of a physical page while typing it or something loosely based on it into a VisionOS document?

First off, you're massively overstating the unnaturalness of the mouse. It's already fully proportional movement pointing, with the added advantages of not obstructing your view with your own hand and being able to rest your hand on a supporting surface instead of having to flail around.

Second, you can already dictate essays, most people don't do that because it's a pain in the ass compared to typing. Drawing is either highly technical detail work which massively benefits from the exact precision inputs of a mouse, or largely about muscle memory which is why artists already draw on high precision pressure sensitive tablets, benefiting from practice in drawing on paper. You're proposing they remove the tablets for no real benefit.

Third, music editing, like programming, if fundamentally a fiddly task requiring talent, practice and an understanding of the field. Both already have a billion different approaches available, someone will probably hack something together for this and someone else will have it as their favorite editor. But on the margin it will turn out that no, in fact there wasn't a massive pool people who are naturals at programming or editing except for the pesky detail of having to learn one of the billion existing options.

I guess you could also code by voice, per the video.

The reason people don't code by voice is that it sucks so many levels of hell. Siri (and other voice recognition agents) can't even get normal text 100% correct right now, what makes them think that shitty voice recognition is going to work properly when I tell it to open and close 5 levels of brackets, use a bunch of non-words like 'var', 'const', 'LazyVStack', 'NSString', and expressions that have non-letter characters in them like '@escaping' and '$0'? Imagine trying to make a regular expression in that and trying to read in "^(+\d{1,2}\s)?(?\d{3})?[\s.-]\d{3}[\s.-]\d{4}$" over voice, or trying to browse the method names in a file like you do with the arrow keys when you briefly forgot what that call and its arguments were (again). Fuck no.

Even that Talon example sounds as incomprehensible as the mouth noises a Vim user emits when they try to describe how having to make 10 different keystrokes is clearly better and so much faster than just selecting the text with the mouse.

Now, this isn't to say that you couldn't do an IDE in VR- in fact, having infinite screen real estate means that you can more easily trace a program's logic down 5 levels where with standard monitors you can only fit 2 or 3 at the same time and the Vision Pro's ability to actually display readable text (I still question that they've actually succeeded- all other headsets except maybe the PiMax just don't have the resolution to do this because a combination of head tilt + low resolution per eye means that text is fundamentally unreadable unless it's so large as to be pointless- so I'd have to see a physical demo of it to fully understand its liimtations).

But considering just how much Vision Pro depends on being a bigger screen with a couple of interesting telepresence applications baked in I really don't think this is going to be as transformative as some might otherwise think, but industry and military applications where you need to keep track of a bunch of interacting systems at once are going to be interested in where this goes since this is kind of just a better Hololens (and those customers are who MS was actually marketing that thing to). I could see a store using these things for "where's this item on the shelf?", an factory providing an auditable inspection pipeline for guaranteeing the condition of a product before it leaves (provided they're dirt cheap to make and replace), and so on and so forth.

Humorously, this is also going to be a test of how well AI does when it comes to adopting a new language- yeah, Swift has some existing support, but the toolkits for visionOS and its UI/UX paradigms sure don't! I guess developers will be learning this one the old fashioned way.

Ever since the computer first arrived, keyboard and mouse has been the standard. You have a flat surface with raised little squares that you smack with your fingers. You have another little rounded shape with a flat bottom you move around, and click with.

my fingers are not exactly small, and this has not been a problem. every few years a company tries to reinvent the keyboard and mouse, and it never catches on. the Logitech trackball mouse is one such example. or ergonomic non-QUERTY keyboards, Wacom tablets, or touchpads (except for laptops).

If you haven't seen the recent demo of Apple's new VisionOS they're breaking brand new ground. The entire OS is built around looking at things, and making minute hand motions to control the icons you're looking at. There are no controllers, no physical interfaces whatsoever besides your eyes and your hands. It's breathtaking to watch.

for one, it costs a lot. Full-immersion headsets have been around for decades and never caught on due to causing nausea or other problems.

This awkward, clunky interface has significant culture war elements, in that an entire class of powerful people arose - specifically people who didn't have traditional status markers like height, strength, or indomitable physical presence. Instead these 'nerds' or 'geeks' or whatever you want to call them specialized themselves in the digital realm. Now, the Zuckerburgs and Musks of the prior generation rule the world. Or if they don't, they soon will.

i dunno where this notion of high-iq people or 'nerds' being small and diminutive came from. it's not backed by evidence either scientific or empirical. given the positive correlation between IQ and height ,we would expect the opposite --for smarter people to be taller, bigger, and heavier. Zuck is short, but Elon and Marc Andreessen are 6'2 and 6'4 respectively. . when i am walking down the street, the people doing construction/road work, by in large, are really short but stocky, but shorter and lighter than businessmen for sure, who i can presume to be higher IQ . same for people at cashiers or other retail or homeless or mentally ill people.

or ergonomic non-QUERTY keyboards, Wacom tablets, or touchpads (except for laptops).

Ahh I forgot to mention this - the QWERTY keyboard is a perfect example of pointless and inefficient input tech.

Can you please stop just saying ridiculous shit without even attempting to justify it?

I realize that I may be outside of the context window of the Motte on this post. I thought folks here would be more familiar with keyboard/mouse history, I was wrong.

Here's a decent writeup of how QWERTY is essentially an anachronistic mistake.

Familiarity with the topic doesn't make it more relevant to your thesis. Dvorak is exactly as arbitrary and hard to learn as Qwerty. The relative efficiency of touch-typing english text matters when you're already doing 100+ wpm and type regularly as part of your job or something. Most people don't touch type, a different layout wouldn't help them any.

Of course, I don't think your thesis has any basis in reality anyway. The average person these days is quite proficient with phone onscreen keyboards, which are vastly worse interfaces than physical keyboards.

And, as with dictation, if you want to do your text input with eyetracking, you've been able to do that on existing computers for years. People keep using keyboards because they are in fact quite good at what they do.

Okay, this is one I’ll back up. I thought it was commonly accepted.

The QWERTY design was intended to minimize typewriter jams. Hitting two adjacent keys too fast tended to jam, because each letter swung a tiny arm. So the design placed common consonant pairs fair apart, maximizing clearance.

In grade school, this was glossed as “intentionally arranged to slow the user down.” I don’t think this is accurate. But there are definitely some features that come straight from the 1870s. Without tiny arms to jam, they’re pure, “pointless” technical debt.

On the other hand, the evidence for any particular replacement isn’t very good. A skilled typist works around the nonsense just fine. So the “inefficiency” is minimal at best. It certainly doesn’t favor nerds! Still, we could in theory benefit more from a different standard.

It isn't efficient (although as you say, it's not less efficient than most other keyboard layouts we've tried) and if he'd just said that I wouldn't have said anything. But calling it pointless is just wrong - there was a point to its design, and the point to it now is that all of the world's fastest typists are fluent in it and converting them would be a monumental task.

That said, we could definitely do better if we could start over. There was a keyboard design for thumbsticks I used to have on the psp that I reached 68wpm on after only a month's usage for example.

Full-immersion headsets have been around for decades and never caught on due to causing nausea or other problems

To put it bluntly, full immersion VR headsets from before the Oculus era (circa 2013) were rather shite.

They had poor software support, terrible displays, and overall were useful for very little indeed.

The hardware has massively improved, especially for any given price point.

i dunno where this notion of high-iq people or 'nerds' being small and diminutive came from. it's not backed by evidence either scientific or empirical. given the positive correlation between IQ and height ,we would expect the opposite --for smarter people to be taller, bigger, and heavier. Zuck is short, but Elon and Marc Andreessen are 6'2 and 6'4 respectively.

Some combination of latent Just World and egalitarian fallacy, that individuals have the same amount of attribute points, just distributed differently.

It’s also sad and darkly hilarious how men can’t escape their height. Men like Cruise and Zuckerberg will forever get clowned for being short, regardless of how rich, famous, and successful they are.

Law of the jungle. You can look at a guy and tell with 90% confidence whether you can kick his ass or not. At some point in prehistory, this mattered a lot. The alpha chimp in "Chimp Empire" had 8 biological children.

Height is one of the biggest factors in this ass-kicking calculation. If I look at a guy like Zuckerburg, my gut says I could kick his ass and steal his girlfriend. (Note: my gut is wrong in this case).

Men will never escape their height. Women will never escape their youth.

While I appreciate a top-level post talking about VisionOS, you seem to be claiming that keyboard/mouse is the moat around high-income tech jobs that keeps out "the strong and physically capable", which seems crazy to me.

Typing speed doesn't meaningfully affect an engineer's productivity (except maybe in the low-complexity world of undergraduate projects). A new operating system isn't going to increase the percent of people who can use threads.

What stops more people from becoming software engineers is that it's hard.

Keyboard and mouse still hold the crown.

It’s shocking to you and me, but most people spend more time using a touch screen than using a mouse and keyboard. Modern UX prioritizes mobile experiences over mouse/keyboard.

What stops more people from becoming software engineers is that it's hard.

Specifically, it's not ability to type, it's having the vision and creativity to actually know how to conceive of something in the first place that most people lack. And neither AI nor fancy headsets can really help with this. Yes, the AI can write code for you, but you need to know what to tell it to write first. You still need to spot pitfalls with its interconnectivity with other parts of the project and know how to smooth them out. Typing faster similarly won't help you if you don't know what you should be typing in the first place.

A machine can assemble a car. But humans still design the cars.

Typing speed doesn't meaningfully affect an engineer's productivity (except maybe in the low-complexity world of undergraduate projects). A new operating system isn't going to increase the percent of people who can use threads.

I don't have research or anything on this, but I'd be willing to bet it's a massive barrier to entry. In programming yes, but most jobs are 'email jobs' where a ton of what you're doing is tying natural language sentences to other people. In those jobs, and pretty much any white collar job that requires communicating, typing speed is important.

Not just typing speed, but the ability to quickly and easily synthesize and write down a chain of thought. I know many people who can speak brilliantly and eloquently, but when it comes to writing or typing down their insights they struggle mightily. Here on the Motte I'd imagine we're heavily biased towards typists and writecels.

Typing speed doesn't meaningfully affect an engineer's productivity

Past some threshold, no. But low typing speed (slower than even your system 2 thinking) certainly impedes productivity. I have a junior in my team who drives me up the wall during 1:1s with her typing speed or rather lack thereof.

keyboard and mouse ... This awkward, clunky interface ... These outdated interfaces

This is an interesting opinion. One I certainly don't agree with and seems in need of justification.

Mouse and keyboard work great. For getting work done they are vastly superior to xbox controllers and Wii style motion controllers. The mouse gives extremely rapid pixel precise selection and movement of graphical objects.

This new Apple tech needs to be jaw droppingly amazing to stand up to the extremely capable mouse and keyboard.

Outside of competitive e-sports, I struggle to see how physical fitness would come to play a significant role in outcomes.

Even in VR/AR, the best way to input text is either speech or typing on a virtual keyboard, or even a real one if the system supports it.

Add in eventual brain-machine interfaces and you won't even need to do anything at all beyond think.

If the latter sounds speculative, the Vision Pro is already a crude BCI, its developers have boasted about being able to track dozens of metrics through cameras and other sensors, especially the anticipatory dilation of the pupil, to predict user input before it's made. It literally reads your mind, albeit imperfectly.

And then there's more explicit stuff that Valve is cooking in the oven, I suspect the successor to the Index will display some.

At any rate, Apple is one of the few companies capable of dragging the future into existence, kicking and screaming in the process, and with a massive price tag attached. Nobody else had the chutzpah to offer a mass market product at 3.5k, even if mainly for businesses and developers, and know the latter are immediately preparing for the consumer version that's going to come out in a year or two. The Hololens never stood a chance, it was born too early with too little to show for it.

Now, I've gone from being a massive VR enthusiast, from the old Oculus prototype days, to being mildly jaded about VR as it currently stands.

I bought a Quest 2, and use it primarily tethered, and have found it's a fun toy worth using for maybe an hour at a stretch before it becomes uncomfortable. In fact, I mainly play a single video game, H3VR, and leave a dozen titles I picked up on sale languishing there because I simply can't be arsed.

I have discovered I am simply too lazy to enjoy most VR, and I prefer playing seated when possible.

In order for me to invest further into the tech, the optics need to get a lot better and the headsets a lot lighter. Perhaps some advancement in peripherals, in addition to hand tracking.

I am aware that most of those things exist, but not all once in a convenient form factor.

In this sense, Apple's product is heartening for the slumbering evangelist in me, as it shows that we'll get there sooner or later. Of course, I'm still terminally lazy, so I'm hoping for BCIs to just let me play by thinking instead of something so quaint as physical motion. Still 5 or 6 years away from that at a minimum.

Add in eventual brain-machine interfaces and you won't even need to do anything at all beyond think.

If the latter sounds speculative, the Vision Pro is already a crude BCI, its developers have boasted about being able to track dozens of metrics through cameras and other sensors, especially the anticipatory dilation of the pupil, to predict user input before it's made. It literally reads your mind, albeit imperfectly.

Yep I strongly agree with this, I see the VisionOS as the first step in the move towards BCI. I just think we already have the technology to at least augment KB+M input in specific areas, so I'm excited to see it happening.

In this sense, Apple's product is heartening for the slumbering evangelist in me, as it shows that we'll get there sooner or later. Of course, I'm still terminally lazy, so I'm hoping for BCIs to just let me play by thinking instead of something so quaint as physical motion. Still 5 or 6 years away from that at a minimum.

Keep that sleeping evangelist alive! If anyone can do it, it's Apple.

Apple only does (good) things years after they've been done by others. They still don't have a foldable, for example. They will not be the ones to first do BCI, if they ever do it.

What are you even on about with this nerds vs jocks schtik. Anything that gives you "power over the digital realm" is not limited by your physical prowess or lack of it. Its limited primarily by your brain !!!

Nerds are high status (the ones that make bank) not for knowing how to use keyboards, but for knowing what to write!!

Its the artists hand that has the magic, not the paintbrush.

Come back when a UI comes out that... doesnt reward higher IQ.

Microsoft demonstrated the same capabilities with Hololens several years ago, so this is not technically groundbreaking. I think Apple will need its usual refinement but also to reduce the cost by about an order of magnitude for this to be a world changing product.

No, Hololens seems positively quaint in comparison to the Vision Pro.

Most damningly, it had a projected FOV of 52 degrees diagonally.

That's like trying to use a large living room TV as a window into the interface, as opposed to Apple who are around 110 degrees from what I've heard.

I suspect the UX is significantly better for Apple, even if both have eye and hand tracking.

I do agree that the price needs to drop, by at least 2/3rds before it becomes a common sight.

I didn't dispute it was better. Hololens 2 was like 5 years ago. I'm just saying it's not groundbreaking, it's Apple doing what they do, letting groundbreaking technology mature and then refining it.

However I'm most interested in how this innovation could shift the balance of power back to the strong and physically capable, away from the nerds.

There's been some amount of this in VR already: BeatSaber is one of the more famous VR games as a whole and can be surprisingly calisthenic (cw: flashing colors), and games like Rumble take that further to a point that can interact with your proprioception.

But there are ultimately games. There are definite advantages to Apple's handsetless approach -- dumbest side, you actually do lose track of where your real hands are when using a lot of VR handsets! ((Though I'm not sure handsetless is the right answer. Lucas's LucidVR tech has that bulky jank only DIY can provide, but cheap haptic feedback allows a lot of things that handsetless can't.)) But it's not immediately obvious that these would have any implications on a 'balance of power' any deeper than the next generation of MMORPGs, or perhaps some corny Pokemon Go-style things.

It's imaginable to see something with broader implications. You don't need to be physically strong to build a house in Minecraft, and you're not doing much more to build a house in VR-Minecraft (though your noodly arms will get a workout!), but there's at least a plausible scenario where XR/AR-HouseBuilding Assistant ends up being a thing. Given the current state of recruiting for the trades (and the people who are recruited often doing awful at it) that's potentially a pretty lucrative design space.

But the reason Software Ate The World isn't that it was something you did with computers; it's that it's something you did once and it applied ten thousand or ten million times without having to redo each one. There are a few places where that matters with just physical strength or dexterity, but most of those spaces are very well-explored.

But if you suppose that a lot of the reason past-gen tech focused on us noodly-armed people was because the active and energetic don't enjoy sitting down at a computer when thinking hard, then maybe that's a different matter.

Ever since the computer first arrived, keyboard and mouse has been the standard. You have a flat surface with raised little squares that you smack with your fingers. You have another little rounded shape with a flat bottom you move around, and click with.

I mean... no? The first computer was the ENIAC, and it didn't even have programmable memory, it has to be hard-wired and manually changed every time you want it to do something different. You didn't interact via keyboard, you interacted via pulling wires.

And the mouse isn't until ~20 years later in the mid 60s. So far as I'm aware, it's first demo'd publicly during the mother of all demos

I mean... no? The first computer was the ENIAC, and it didn't even have programmable memory, it has to be hard-wired and manually changed every time you want it to do something different. You didn't interact via keyboard, you interacted via pulling wires.

Ok fair, this was a bit sloppy I'll admit.

Fascinating tech.

I'll take your bet on the "balance of power," though. Why would you think having a better OS would punish the physically weak?

Actually, what is it about current interfaces that you think favors the unhealthy or the dysfunctional? Nerds aren't weedy as an adaptation to crawling around in the code mines. They're spending time hacking and playing video games instead of working out. Or sitting at a desk instead of lifting boxes. Augmented reality isn't going to bridge that gap any more than standing desks.

No longer will clunky interfaces make sense… Ideally the most intuitive and common control schemes will reward physical fitness and coordination.

Do you have any examples that you could share? It’s hard for me to visualize what you’re thinking of here.

I think existing computer interfaces are pretty good. I wouldn’t call them clunky. The vast majority of what most people do with computers involves reading and writing text, and watching video. It seems to me that those interfaces are already pretty optimized and there isn’t much more utility that you could squeeze out of them. I don’t think people think “this is clunky” when they’re reading (old.) reddit or watching youtube.

The vast majority of what most people do with computers involves reading and writing text, and watching video.

Thinking about my own habits, this is true for me. I consume a lot of text. That process includes a lot of detailed reading, skimming, and skipping over sections. The pace at which I scroll is highly variable, depending on the source material and the layout of the site/document in question. To me, I think the key question would be the extent to which I can replicate this process (to at least make VisionOS not worse) or somehow improve upon it.

I'm breaking my self-imposed Motte break to link this Ross Scott video about OS GUI design. We could already improve regular "flatscreen"/"pancake" UI without needing to go all-in on AR/VR, though I suppose the temptation of Apple is "why not go all the way?"

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats