Steve Troughton-SmithOct 29

Today I received an incredible surprise care package containing a 1TB Apple Vision Pro (M2), official case, weighted ResMed dual strap, and developer strap, all for which I am incredibly thankful. 🤯

Finally, after 2 years, I will be able to dive in to visionOS properly*

(And no, it wasn't from Apple)

*in as much as is possible in a non-supported country, which Apple has made /as painful as possible/, but I will complain about that in due time

Show threadSteve Troughton-SmithOct 29

My custom satellite TV app works great on visionOS 👌

I can pick up a 70" virtual tv screen and wander round the house with it, or bring it to the moon or a desert island

Show threadSteve Troughton-SmithOct 29

One aspect of the Vision Pro experience that really fucking sucks is the Apple TV app.

Not because it's bad.

Because its contents are completely unavailable if Vision Pro isn't supported in your country.

That means no Apple shows, no Paramount+, no nothing. Deep links go nowhere.

You can sometimes play these TV shows and movies using the Apple TV website (but not always), but that's it.

The Apple TV experience is 75% of the value of a Vision Pro, and it's just not available to me in Ireland

Show threadSteve Troughton-SmithOct 29

I can understand the App Store not being available (though I disagree with that, too), but to lock away my access to the services I'm already paying per month for is a bit of a gut punch — especially when it's some of the only content Vision Pro *has* right now.

(I knew this going in, for the record, but now I've experienced it I want to share how I feel about it)

Show threadSteve Troughton-SmithOct 29
In other news, visionOS is *gorgeous* in practice. It’s easy to forget that when discussing it from afar
Show threadSteve Troughton-SmithOct 29
If you think Apple's immersive environments are cool, try Disney's. The Alien: Earth one is fucking terrifying. It genuinely feels close to real life in quality — no uncanny valley here. That's the first VR I've seen that I could believe
Show threadSteve Troughton-SmithOct 29
It’s kinda shocking to see just how few key developers support visionOS. For example, 2 years on and Dropbox is still completely unavailable on the platform. Good luck having productivity apps if you can’t access your files
Show threadSteve Troughton-SmithOct 29
Speaking of shocking, the visionOS 26 shell crashes to an Apple logo an alarming amount of times per day. It crashed while I was dictating audio, it crashed when I entered an immersive space in Disney, and it crashed when I tried doing a FaceTime call. I could tell the OS was in rough shape from the Simulator, but this is deep into unfinished territory
Show threadSteve Troughton-SmithOct 29
The displays in the Vision Pro are effectively perfect; I don’t think they would benefit at all from more pixels. What the device could really do with are much better cameras and optics. A variable focal distance would go a long way to making this feel great. The passthrough cameras though need to be 10X better for anybody to take them seriously. I still don’t know how Apple convinced pundits who tried this at WWDC23 that passthrough was as good as they claimed. Y’all need help
Show threadSteve Troughton-SmithOct 29
Passthrough vs in-headset content. It handles low light better than the Quest 3, but it's still not great
Show threadSteve Troughton-SmithOct 29

I’ve had the headset on for maybe 8 hours today, comfortably. It is so much heavier than the Quest 3, but there is effectively nothing in the OS that can make you motion sick…

…except for ‘Immersive View’ of spatial video. An optional mode in Photos, hidden behind a button — a button I would recommend never ever pressing, because only a couple seconds of that can make you very sick 🤮 Use it for stabilized fixed-perspective content only

Show threadSteve Troughton-SmithOct 30
One very zen thing to try on visionOS is to lie on your back and look at the stars in the moon environment at night
Show threadSteve Troughton-SmithOct 30

This much was obvious before, but Vision Pro has a massive input problem: text entry and editing *sucks*.

I still think the platform would benefit greatly from a combined keyboard/trackpad accessory, purpose-built, and that's the kind of thing Apple could and should build

Show threadSteve Troughton-SmithOct 30
Something that slipped my mind is that you can AirPlay iPad and iPhone screens to the Vision Pro, which is kind of cool. Sadly, it doesn’t support any kind of touch handling
Show threadSteve Troughton-SmithOct 30
I do wish you could capture multiple Personas and swap between them at will. Business, casual, etc
Show threadSteve Troughton-SmithOct 30
It is abundantly clear that Vision Pro is a developer kit, and the software and platform should have a 'beta' label. There are many big problems that need to be solved, and they will still take years of effort. Should it exist? Absolutely. There should be a high-end headset, a 'cheap' headset, and whatever happens with glasses — these are all different products, and they'll drive the ecosystem forward in different ways. Above all, Apple, start making peace with developers. Give them kits, and $
Show threadSteve Troughton-SmithOct 30

Every Apple Developer should be able to log into the developer site and order Vision Pro hardware, light seals in any size, and the developer strap. In every country.

It boggles the mind that they haven't taken this step already

Show threadSteve Troughton-SmithOct 30

FB20871373 — Every developer should be able to order Vision Pro from the developer site, in any country

#radars

Show threadSteve Troughton-SmithOct 30

I had a lot of complaints about the original version of visionOS, but they have addressed pretty much all of my most-critical feedback in the OS releases since. If they actually had a good-enough relationship with developers to fill this thing with software, it would be an incredible computing device.

With an empty App Store, though, it’s still merely an expensive toy. There are so many ways they could have fixed that in the time since launch, Apple just need to get over themselves…

Show threadSteve Troughton-SmithOct 30
3D RealityKit content in visionOS is kinda shocking. It feels real, completely believable. Like there’s a physical object floating in front of me. The way it reacts to light is wild, and how it reflects the view out of my (irl) window. I am shook
Show threadSteve Troughton-SmithOct 30
The more time you spend in visionOS, the more you fool your brain into thinking that these floating windows are physical objects in real space. The visual quality is that good. It's like a superpower, being able to summon 'UI' everywhere you go
Show threadSteve Troughton-SmithOct 30
I’m using Moonlight, which works pretty well, but I really wish I had Windows Virtual Display on visionOS as well as Mac Virtual Display so I could have both my Windows and Mac monitors side by side like in real life
Show threadSteve Troughton-SmithOct 30
I really love visionOS 26's ability to snap and align windows to flat surfaces, like walls, or monitors. Makes it really easy to replace my real displays with virtual displays
Show threadSteve Troughton-SmithOct 30
Seeing your own apps floating in augmented reality is a deeply emotional experience. I made this? Holy shit.
Show threadSteve Troughton-SmithOct 30
I made this? I made this
Show threadSteve Troughton-SmithOct 30
I’m really falling in love with visionOS, you guys
Show threadSteve Troughton-SmithOct 31

The Disney environments make me wonder what Pixar could do if they made a 'movie' (short, stage play, etc) entirely in RealityKit. The Monsters Inc Scare Floor makes me feel like I'm *in* the movie. Not in VR.

There is a level of fidelity here that's impossible to describe without being there yourself. No pixels, no discernible textures or polygons. You're just… there

Show threadSteve Troughton-SmithOct 31

Testing my apps from the top of a mountain feels kinda profound 😂

Speaking of which, I did bringup on my Broadcasts 4 codebase on visionOS, which was something I wasn't planning anytime soon

Show threadSteve Troughton-SmithOct 31
Not me wishing the visionOS moon environment actually had a moving, accurate starry skybox shader that syncs with the time of day. You can tell it's a flat texture, which is unsurprising
Show threadSteve Troughton-SmithOct 31

I've also tried to summon the visionOS Control Center via gesture in real life, or things from across the room 💀

This is the new forgetting your Mac doesn't have a touchscreen

Show threadSteve Troughton-SmithOct 31

I know plenty of people have complained about it, but it's a little infuriating to do visionOS development with Mac Virtual Display, because every time you build and run the app is spawned where you're looking (i.e. on top of the Mac) instead of where you last carefully positioned it.

I really hope they anchor that between runs in the future

Show threadSteve Troughton-SmithOct 31
Speaking of development — I'd kinda forgotten, but I have multiple [near-]release-quality apps ready to go for visionOS that I just never shipped, and actually seeing them here in-headset kinda gives me the urge to go for it
Show threadSteve Troughton-SmithOct 31
One area that visionOS really needs to improve in is occlusion. It's good, but it's not great, and it can be very distracting. Any time a virtual window clips objects in the real world, you're taken right out of the illusion. I envision a future version of the software that uses gaussian splatting perhaps to precisely mesh out your room to make this much better
Show threadSteve Troughton-SmithOct 31
There's a genuine disappointment at being back in the real world after using and working in visionOS for a few hours 😂 You get really used to being able to summon apps everywhere you go, and grab and fling windows to/from the next room over
Show threadSteve Troughton-SmithNov 1

Some of Apple's visionOS sample code is more evocative than others

This is Canyon Crosser

https://developer.apple.com/documentation/visionos/canyon-crosser-building-a-volumetric-hike-planning-app

Show threadSteve Troughton-SmithNov 1
Now this is multiwindowing
Show threadSteve Troughton-SmithNov 1

Vision Pro battery life is so short that there's no time to have battery anxiety. If you're sitting at a desk, you probably want it plugged in in your pocket. You have enough time to walk around the house a bit, maybe cook a meal (which I've done, of course), but then it's wall time again.

Fortunately it charges pretty quickly, and doesn't discharge if used while plugged in.

Granted, my unit has a 2 year old battery, but somehow I doubt it's been used continuously for 2 years 😂

Show threadSteve Troughton-SmithNov 1
I had thought visionOS' guest mode might actually have its own user data if not a full-on account, but no it's just a temporary hand/eye config, an app ID filter, and automatic mirroring. All your data are belong to guest
Show threadSteve Troughton-Smith

So, so much of the visionOS experience could be vastly improved if Apple just convinced Netflix and YouTube to release their native apps.

Two companies.

Apple, you can figure out a deal with two companies, surely? Don't you have a master negotiator SVP? Why would you spend 7 years and billions of dollars developing a product and not be able to convince two partners to ship apps they already have?

Nov 1 at 1:52pmIvory for Mac
Show threadSteve Troughton-SmithNov 1
Speaking of visionOS experience, SurfBoard (the visionOS shell) continues to go down multiple times a day on 26.0.1. I'll have to see if 26.1 fixes that next week, but right now it feels like there are fatal memory leaks in the OS that can bring it down way too easily. It happens more often with RealityKit scenes and 3D content, but not always. And not all apps/widgets come back up where they're supposed to, if at all
Show threadSteve Troughton-SmithNov 1
Granted I have been using visionOS for like six or more hours a day all week, and that tends to highlight things like memory issues…
Show threadSteve Troughton-SmithNov 1
visionOS feels more real than real life
Show threadSteve Troughton-SmithNov 2

I happened to be logged in to my US account on visionOS and I came upon a link to the immersive Metallica concert, so I gave that a try…

Probably the best concert experience I will ever have in my life. Absolutely incredible.

It's a mix of stage close-ups, on-rails panning drone shots, crowd close-ups, arena shots, and some vignettes. You could literally count the hairs on James Hetfield's arm, reach out and touch the plectrums on the mic stand, or recognize every phone model in the crowd

Show threadSteve Troughton-SmithNov 2
I listened to a lot of Metallica in my youth; there's a moment where a member of the crowd gets to sing close up to the band, and then the focus shifts elsewhere, but if you linger on the guy in the audience instead you see him collapse with emotion as everybody around congratulates him. Having something preserved like that in such a high fidelity brings a tear to your eye
Show threadSteve Troughton-SmithNov 2
I do want to know how many times that poor lady had to cross that highline so that Apple got all the camera angles it needed 😅
Show threadSteve Troughton-SmithNov 2
As easy as it might be to speculate on the demise of Apple's AR/VR ambitions based on the sales numbers of the Vision Pro, actually using it, seeing what they've done, seeing what they're still doing behind the scenes in both the media production and in the developer space, I think this platform is going to be around for a very long time. Using it feels profound, and that'll only become more obvious as it gets more accessible in future form factors and price points
Show threadSteve Troughton-SmithNov 2

To use Vision Pro optimally, you need an empty room with a standing desk, with only a keyboard and trackpad on it. And great lighting 😂

I am definitely not so lucky

Show threadSteve Troughton-SmithNov 2
Apple's AI-spatialized photos in visionOS 26 look so convincing that you no longer need actual Spatial Photos to exist. They're almost like real life
Show threadSteve Troughton-SmithNov 2
Also something that’s incredibly impressive is the spatially-consistent window anchoring, even in vast outdoor spaces. I can place a window somewhere outside, then walk around the garden with the dog, and when I come back to my original position the window is right where I left it
Show threadSteve Troughton-SmithNov 2
Not to say it’s perfect, of course. Widgets tend to drift by a couple inches in between sessions, and like to levitate off the walls. But still
Show threadSteve Troughton-SmithNov 2

Widgets on visionOS can work very well as app launchers that you can keep around the house rather than leaving app windows open. Apps spawn right on top of their widget when invoked.

This kind of widget isn't something apps do by default, though, but now that I've realized the benefit I think it's something I should plan to include for all of mine.

The smallest widget size though is still 3x too big — about 40x40cm — so you can't exactly have a bunch sitting on your desk

Show threadSteve Troughton-SmithNov 3
The outward-facing display on Vision Pro does serve a purpose — the Persona capture UI is particularly impressive. But it certainly didn't need to be the expensive holographic monstrosity it is now, and it doesn't need to ever show your eyes. I feel like you could replace it with a flexible OLED instead and save a ton of hardware
Show threadSteve Troughton-SmithNov 3
First software update LFG!
Show threadSteve Troughton-SmithNov 4
It's really hard to describe how special visionOS is to use to somebody who hasn't had time with it with their own apps, their own content. I feel emotions using this device that I haven't felt since the early days of the iPhone and iPad. And I was a comparative visionOS 'expert' even before touching the device. I lose all awareness of the headset itself, the black borders around the FOV. The OS just becomes part of my world
Show threadSteve Troughton-SmithNov 4
Remarkably, you can use Vision Pro in pitch dark, and lying down in bed, the 3D sensing it has is that good. It even shows your hands and fingers in 'ghost' form, even though you can't actually see them with your eyes (or the cameras). Emits no light to bother anybody else. It makes for a great nightstand computer
Show threadSteve Troughton-SmithNov 4
I spent some time going through the far reaches of my photo library, 20+ years ago, and hitting the 'spatialize' button in the AVP. On photos taken with a DSLR or film camera, it's like getting a physical window back into your childhood. With the perfect camera angle, and when the quality is high enough, it actually does feel like real life, like stepping right back into your memories
Show threadSteve Troughton-SmithNov 4
I know I spent a lot of effort building it, and it has no users, but I am struck by how nice Broadcasts for visionOS is on an actual headset. I love that the audio 'emits' from the mini player window in 3D (I wish Apple's Music app did that), and where I used subtle 3D visuals. All the multiwindowing works great. I designed all of it without access to a device, and I can see now, 2 years on, that I made a lot of the right choices. Sadly it never made it to the front (and only) App Store page
Show threadSteve Troughton-SmithNov 4

Is Mac Virtual Display a killer app?

I mean, kinda. I wish it actually showed my (non-Apple) keyboard more than 10% of the time. And it does have moments where it refuses to connect without a reboot. Also it doesn't save its anchor point or window size, which is kinda infuriating.

But other than that, it's kinda awesome. Latency and quality feels near-perfect — it looks just like my 4K display, sharp as anything at 1080p@2x

Show threadlawbeckeNov 4
@stroughtonsmith I have next to no use for or interest in a VP as it stands, but a stripped down version intended to be used as a display would be much more tempting.
Show threaderin sparlingNov 4
@stroughtonsmith This is the hardest thing about demoing it for friends. The novelty wears off, but ... how do you demo a productivity tool that's tailored to JUST you?
Show threadHannes GüdelhöferNov 4
@stroughtonsmith I find it breaks down if you start opening Videos (YT for example). But I agree, the ultra wide setup is a great replacement for an external monitor! (I have my gripes with not having the option of 3/2 independent windows, because it does not work with fullscreen mac apps. Which is exclusively what I use normally, but it's very apple to make it incompatible with that ¯\_(ツ)_/¯)
Show threadJohn AppleseedNov 4
@stroughtonsmith I was just talking with my wife yesterday that MVD is 100% the killer app for ME. Can’t speak to the market as a whole, but when I’m using my Vision Pro consistently, it’s because I’m using it as a giant ultra wide display for my 14” MacBook Pro. It’s my main use case 98.9% of the time.
Show threadShaun TurnerNov 4
@stroughtonsmith I had to use MVD for 8 hours a day for a week while my Studio Display was in for repair on a cracked screen. It worked great.
Show threadJosh HrachNov 5
@stroughtonsmith It also rarely shows Apple’s own Magic Keyboard and Magic Trackpad in immersive. It’s super annoying.
Show threadBoxOfRainNov 4
@stroughtonsmith
I noticed even on the iPad it works remarkably well with some DSLR photos.
Show threadTaylor HaddenNov 4
@stroughtonsmith To anybody thinking Steve is off his rocker, the cool-aide really is that tasty.
Show threadJosh HrachNov 4
@stroughtonsmith I can’t imagine a world without Spatial FaceTime as an option.
Show threadMarkVNov 3

@stroughtonsmith I think of it like the TouchBar. A not-terrible idea that was worth shipping to see how it would land in the real world. They have to be willing to ship a few risks.

I also hope they see it’s turned out to be a dead end, and drop it in the next version to save on weight and cost.

Show threadMarcello Bastéa-ForteNov 2
@stroughtonsmith loving your speed run of experiencing and understanding Vision Pro for the first time almost 2 years later, feels like reliving memories haha. I’d be curious if in a few months where your thoughts settle
Show threadJohn DavisNov 2

@stroughtonsmith if you enable some of the Xcode debug features you can see how the world mesh updates in real time :D.

I notice my widgets “settle” into position when I put my headset on after a while.

Show threadBuckNov 2
@stroughtonsmith This may be, but they are so far out of the realm of where the price needs to be (for mainstream uptake), that I'm not hopeful for it anytime soon. That also discounts form factor drawbacks that will always be there. I know absolutely nobody who does anything with a Quest, and it has nothing to do with price or content. They just don't want to spend time in a VR headset. They try it .. it's cool... and they don't want one. 🤷‍♂️
Show threadMarkVNov 2
@buck @stroughtonsmith How would you feel about that price point if they turned the “Mac virtual display” feature into an embedded virtualized Mac? So you’d basically get a Mac, an iPad, and a headset for that same price?
Show threadSteve RigginsNov 2
@markv @buck @stroughtonsmith the first Mac 128k was $2500. With inflation, that is $7800. It had little to no software. Hardly any memory. 1 bit black and white. Yet it was amazing for the time.
Show threadBuckNov 2
@steveriggins @markv @stroughtonsmith Price is but one issue. The Mac was never "head mounted".
Show threadSteve RigginsNov 2
@buck @markv @stroughtonsmith right, it was 17lbs, way too heavy for the head :)
Show threadBuckNov 2
@markv @stroughtonsmith Still the same problems of living inside a scuba mask. I know this is hard for some among us to understand, but a lot of people don’t find the form factor enjoyable.
Show threadGabriel NNov 2
@stroughtonsmith I was reading the entire thread just to try the get the context of this comment. No luck.
Show threadSteve Troughton-SmithNov 2
@wtrmt it’s in reference to one of the original and most well-known demo videos for the Vision Pro
Show threadThom, exceedingly pureNov 2
@stroughtonsmith Does that require a €3500 boondoggle, though?
Show threadSteve Troughton-SmithNov 2
@thomholwerda today, sure. But it'll look just as good in 20 years
Show threadThom, exceedingly pureNov 2
@stroughtonsmith So will any 4K or 8K recording.
Show threadSteve Troughton-SmithNov 2
@thomholwerda really no, this is very different to other VR stuff I've seen. And for good reason — it required a shitload of expensive bespoke hardware (and software) to pull off. Of course, now it has been developed anybody will be able to benefit from it
Show threadShantaraNov 1
@stroughtonsmith If you die in visionOS, you die in real life
Show threadsfo_laxNov 1
@stroughtonsmith this thread is pushing me towards buying a new Vision Pro. Resist.
Show threadTaylor HaddenNov 1
@sfo_lax @stroughtonsmith Despite all of its obvious shortcomings, it's pretty great. If you can get all of your computing done on an iPad with a keyboard & mouse, the Vision Pro is kind of just better (with that same keyboard and mouse).
Show threadFabienNov 1
@stroughtonsmith I have serious fears that once the tech become ubiquitous, people will neglect the world even more as they’ll be able to hide/decorate it virtually.
Show threadDimitri BouniolNov 1
@stroughtonsmith Like someone else said, I hear the reboot chime daily even when not in use, which makes me think its not _just_ RealityKit scenes haha
Show threadThom, exceedingly pureNov 1
@stroughtonsmith Why waste manpower on like 5 users?
Show threadSteve Troughton-SmithNov 1
@thomholwerda Apple has a big pocket book
Show threadThom, exceedingly pureNov 1
@stroughtonsmith Ballroom has higher priority.
Show threadD. Griffin JonesNov 1
@stroughtonsmith What I like about using YouTube the website is I can run Vinegar, which replaces its player with a visionOS native video player
Show threadNico ReeseNov 1
@stroughtonsmith Back in the day they fought to get the Beatles on to iTunes and made a big fuss about it. I would argue getting Netflix & YouTube (plus Netflix in the TV app in general) is the bigger deal.
Show threadJorge Salvador CaffarenaNov 1
@stroughtonsmith Apple should have to make a deal where they develop the apps themselves, I don’t see google or Netflix spending resources for a device that has dozens of users, in just couple countries.
Show threadJadeNov 1
@stroughtonsmith Netflix still doesn't integrate with the Apple TV app watchlist. It seems like if Apple were cutting checks/deals, that problem would have been solved years ago.
Show threadMarkVNov 1
@stroughtonsmith Given how much Apple has proven its willingness to screw over developers and compete against them for service revenue once they’ve been able to grow their platform enough to lock in the customer… why would anyone help Apple become as dominant in headsets as they are in smartphones?
Show threadRavi Nov 1

@markv @stroughtonsmith Exactly.

 just doesn’t understand quid pro quo. They’re too used to having everyone ask how high to jump.

Show threadGKNov 1
@stroughtonsmith And Google Maps. Oh I need immersive Google Maps so badly