The screen sharing is the most exciting part (for me, anyway): we're maybe less than a year away from opening the ChatGPT app and being able to share screen, hand control over, and tell it to get your project compiling while you go grab a coffee or something. I'm surprised Microsoft hasn't jumped on this, with OS-level support you could expose a lot of context and possible interaction directly to the LLM without relying on vision (but I guess if vision is good/cheap enough then that's not necessary!)
I use Copilot a lot now that it's just natively there at all times, but it has basically zero ability to interact with the OS (I think all it can do is rearrange windows, pop up some settings, and read Edge web page content). I'd like it to have a "shared command prompt" that it can autonomously type commands into when I tell it to do things. The pasting back and forth is annoying.
p
The screen sharing is the most exciting part (for me, anyway): we're maybe less than a year away from opening the ChatGPT app and being able to share screen, hand control over, and tell it to get your project compiling while you go grab a coffee or something. I'm surprised Microsoft hasn't jumped on this, with OS-level support you could expose a lot of context and possible interaction directly to the LLM without relying on vision (but I guess if vision is good/cheap enough then that's not necessary!)
I use Copilot a lot now that it's just natively there at all times, but it has basically zero ability to interact with the OS (I think all it can do is rearrange windows, pop up some settings, and read Edge web page content). I'd like it to have a "shared command prompt" that it can autonomously type commands into when I tell it to do things. The pasting back and forth is annoying.
More options
Context Copy link