Steve Troughton-Smith
I do think Apple is going to be in a tough spot with game devs for visionOS. Sessions make it clear that hand tracking just won’t be accurate enough for anything fast-paced, and everything has to be re-hosted using Apple APIs to do even basic things with the system compositor. I expect there will be a rising chorus of ‘we need hand controllers’ at the very least. If Apple doesn't have hand controllers of some kind by version 2.0, it could negatively impact the platform for years to come #WWDC23
Jun 7, 2023 at 1:39pmIvory for Mac
Show threadSteve Troughton-SmithJun 7, 2023
The real worry is that gaming is such a blind spot for Apple that they don't realize this at the decision-making levels, and won't have been preparing for this eventuality. You don't just throw together hand controllers at the last minute — they need to do similar inside-out tracking to the headset itself, with multiple cameras and sensors. This may be a painful wake-up call
Show threadSteve Troughton-SmithJun 7, 2023
Let's not think about the fact that Apple burned all its bridges with Epic, whose Unreal Engine, this past generation, has become the only game engine in town with which anybody wants to work. Tight Unity support is great, but not in any way enough
Show threadCesare ForelliJun 7, 2023
@stroughtonsmith Brought to you by the company who fought for a decade with Nvidia, when everybody wanted they GPUs…
Show threadThom HolwerdaJun 7, 2023
@stroughtonsmith Apple's feud with NVIDIA set the Mac back, graphics-wise, by at least a decade, if not more. Repetition?
Show threadMiguel de Icaza ᯅ🍉Jun 7, 2023
@thomholwerda @stroughtonsmith yet Xbox and PlayStation have been on amd for decades. Not as big of a moat as people claim.
Show threaddonkeyJun 8, 2023
@Migueldeicaza @thomholwerda @stroughtonsmith And also not solely Apples fault like is being implied. There’s plenty of blame both sides of that argument.
Show threadLucas MengeJun 7, 2023
@stroughtonsmith I can’t stop thinking that:
- Apple reaches out to Epic to partner on the headset a few years ago
- Epic has the power, lobbies for alternate App Store/payment methods
- Apple says no
- Epic gets annoyed and pulls the stunt they did thinking “are they really gonna wanna lose the biggest game engine from their platform and upcoming amazing headset?”
- And here we are
Show threadJanJun 7, 2023
@stroughtonsmith „Apple burned all the bridges“… who again submitted an update they knew would be rejected and had everything ready to sue the next day? 🤔
Show threaddonkeyJun 8, 2023
@meaningful_jan @stroughtonsmith Indeed. Bridges were burned. But I don’t believe Apple lit the fire here.
Show threadDΛNIEL TʘMΛNJun 7, 2023
@stroughtonsmith was it Apple that burned the bridges, or Epic? 🤔
Show threadTim JohnsenJun 7, 2023
@stroughtonsmith feels like it was Epic doing the bridge burning
Show threadMaxJun 7, 2023
@stroughtonsmith This does not mean that Epic will not add support.
Show threadRob McDougallJun 7, 2023
@stroughtonsmith as someone who, by all rights, should have a gaming PC to do his job, Apple's major weakness right now is all gaming related. The M2 GPU's are excellent for laptops, but the desktop is only so-so, compared to an NVidia 4090 which, for the money you'd pay for a Mac Pro, would be what you'd expect.
Show threadZhuowei ZhangJun 7, 2023
@stroughtonsmith I can find _zero_ AR experiences built in Unreal.
Every single app in https://vrscout.com/news/five-cool-mixed-reality-games-apps-for-quest-pro/ is Unity, as far as I can tell.
Even most VR experiences on Oculus Quest are Unity.

(According to Unity themselves, they power 90% of AR experiences on headsets: https://create.unity.com/unity-pro-for-industrial)

I don't think burning bridges with Unreal will actually hurt Apple's headset, especially when they're not marketing it as a VR/gaming device.
Five Cool Mixed Reality Games & Apps For Quest Pro - VRScout

Meta's latest VR headset already features a solid lineup of mixed reality games and apps, from Figmin XR to Puzzling Places.

VRScout
Show threadGrievous AngelJun 7, 2023
@stroughtonsmith they had games controllers in the demo video?
Show threadNamanJun 9, 2023
@grievousangel @stroughtonsmith Yeah but not motion controllers. The existing games like Beat Saber need motion controllers.
Show threadNamanJun 9, 2023

@grievousangel @stroughtonsmith I do want to add that games like Beat Saber came about *because* motion controllers were the only input mechanism.

Hand-tracking and “traditional” controllers might enable different new kinds of games.

Show threadNamanJun 9, 2023
@grievousangel @stroughtonsmith The problem with hand-tracking is that it can’t be used for any quick motion tracking which is needed for gaming. Fast movements are hard for the cameras to track.
Show threadGrievous AngelJun 9, 2023
@nmn @stroughtonsmith yes. 2 different gaming models there. Vision probably lacks the GPU to push enough polygons for pc style gaming. But good for lightweight post iPad ar gaming.
Show threadSteven Op de beeckJun 7, 2023
@stroughtonsmith I believe it was Epic that struck first.
Show threadColJun 8, 2023
@stroughtonsmith the decision to not support Flash on the original iPhone springs to mind. Flash was also the only game in town… we’re at the start of a process…
Show threadBlai PratdesabaJun 7, 2023
@stroughtonsmith real gaming has always been an afterthought on the Apple ecosystem, never part of the core development of their products.
Show threadRobert Petersen Jun 7, 2023
@stroughtonsmith yep, most gaming without controllers is gonna be a non-starter on Vision. You got to have some tactile interaction and haptics.
Show threadDallas GrootJun 7, 2023
@stroughtonsmith I see ideas for v2
Show threadBenRiceMJun 7, 2023

@stroughtonsmith I’ve talked a few dev relations people about this; how much I love Beat Saber, and how it’s dependent on having a controller in each hand giving you haptics and a sense of physicality

At the very least they seem to really appreciate the feedback

Show threadOwenJun 7, 2023
@stroughtonsmith Hand controllers suck! Apple must find another way!
Show threadSteve Troughton-SmithJun 7, 2023
@vanitalo when there are use cases that require holding physical objects and interacting with multiple buttons, there is no other way. Just delusional design
Show threadOwenJun 7, 2023

@stroughtonsmith This is why game developers need to rethink game controls. There will be more natural ways to interact when moving to VR.

Needing to use physical buttons and button combinations is not the way forward… it detracts from the user experience!

Games have become visually amazing from a graphical standpoint, but IMO the UX is still pathetic. We need technological improvements that enable immersive natural control experiences to really take games to the next level! 😎

Show threadSteve Troughton-SmithJun 7, 2023
@vanitalo they won’t. Not only will game devs simply not consider the platform, but no amount of waving your arms in the air can replace precise, tactile physical controls no matter how much technology you throw at it. We have a decade+ of iOS to prove that even touch isn’t enough
Show threadOwenJun 7, 2023

@stroughtonsmith The problem is limited thinking. It’s not an easy problem to solve, but it is solvable.

Until then, I’ll largely continue to ignore gaming. My standards are too high to accept what people are currently churning out and I won’t compromise.

It’s all or nothing for me and I’m fine with that! 😎

Show threadConorJun 7, 2023
@stroughtonsmith Apple
Is going for #herfilm not #MetaQuest.
Show threadMichaelJun 7, 2023
@stroughtonsmith the story of the AppleTV 🙄
Show threadRGJun 7, 2023
@stroughtonsmith They seem so averse to first party controllers. How much gaming on tvOS suffered after its introduction because they insisted on using the remote as the controller.
Show threadmingistech Jun 8, 2023
@stroughtonsmith Though far from ideal, and I completely agree regarding the importance of inside out tracking on modern VR controllers, the Nintendo JoyCons might bridge this gap. With built in support on macOS, iOS, iPadOS, and tvOS, it’s likely the Vision Pro will support the JoyCons. The Vision Pro having advanced sensors for hand positioning could easily track JoyCon position and the JoyCons accelerometers and gyroscopes can provide additional details. Again, not ideal.
Show threadCraig GrannellJun 7, 2023
@stroughtonsmith Vibes of “if you want to make games for Apple TV, you have to support Siri Remote” and those awkward shots of people pretending they were having fun playing Asphalt using that atrocity of a controller.
Show threadKeirJun 7, 2023
@stroughtonsmith you can pair an Xbox or PS5 controller to it though which would probably be better for most games (although anything like Beat Sabre wouldn't be possible with those or your hands I suppose!)
Show threadSteve Troughton-SmithJun 7, 2023
@keir you can play legacy games. You can't play VR games, no
Show threadKeirJun 7, 2023
@stroughtonsmith yeah :/
Show threadjimshredsJun 7, 2023
@stroughtonsmith AR turn based table top game would be next level tho.
Show threadMichaelJun 7, 2023

@jimshreds @stroughtonsmith something like this? They are also working on supporting hand tracking for this game, definitely seems possible.

https://youtu.be/gxWzjgoNHl8

Demeo is Now Available in Mixed Reality!

YouTube
Show threadjimshredsJun 7, 2023
@mi @stroughtonsmith def but with Hellfire Club.
Show threadCaleb Hearth Jun 7, 2023
@stroughtonsmith It’s a shame we don’t all have some kind of accelerometer strapped to our wrists that could be used for this.
Show threadMichaelJun 7, 2023
@stroughtonsmith I‘d love to have M2 performance for standalone VR gaming, but if Apple focuses on productivity and a more general purpose device that’s going to help VR/AR more than yet another gaming headset. For these use-cases, a controller might just be in the way.
Show threadjimshredsJun 7, 2023
@stroughtonsmith Railroad Tycoon in your living room. Also brings that Apple patent for Smart Ring back into view.
Show threadGernotJun 7, 2023
@stroughtonsmith they could just add support for the PSVR2 controllers, which are very good.
Show threadRickJun 7, 2023
@stroughtonsmith I wonder if they’re considering it a third-party opportunity.