On Game Development for the Apple Vision Pro

I’ve been interested in game development for a good portion of my life. Programming has been a part of my life for as long as I can remember. I’ve tried diving into so many different environments, languages, engines, libraries, and I’ve been able to make pretty good sense of them all. From wanting to spew garbage collectors in C++ to serving delicious cookies in PHP, there hasn’t been a mountain high enough of a valley low enough to shake my intrigue to learn something new. Despite the years and years dedicated to learning and tinkering, I don’t have much to show for it. There are so many projects lost to time and data corruption, but they don’t haunt me. They’re a part of my journey. Whether lost or forgotten, they’re pieces of what has honed my mind into the logical thinking machine I wear in my skull. I’d like to change that though. I think I might enjoy having something to show for the endless endeavors hunched over a keyboard clickity clacking away, scrolling documentation, watching tutorials. I’d like to let my interests bloom into innovation, enjoyment, productivity. I’m not in this to make a get rich quick app, I don’t want to rake in cash selling loot boxes. I don’t really want to monetize my creations at all. I want to make experiences for myself, and share them with the world.

At this point in time I’m quite heavily invested in the Apple ecosystem. This transition lead me to wanting to learn Swift earlier this year. So I have familiarized myself with Xcode and am reading the official Apple Swift Fundamentals books. It is an interesting language. It is different. Different in ways I’m not sure I even fully understand yet. I am happy I spent time learning Python, I think having that knowledge has aided me in understanding the form and function of Swift a bit. Python is worlds away from Swift though. Python is as open, robust, and organized as you want it to be or not to be. Swift is stern, and demanding of structure. Regardless, I like it. The strict syntax and structure ensures your code will always work properly, securely, and efficiently.

A good chunk of reason I wanted to learn Swift however rested in my interest in the Vision Pro. I somehow managed to turn myself from “I’m never going to be able to get one of these” to “I think I’m on my way to go get one right now” in a matter of months after it released. It was a terrible financial decision, but a wonderful investment in my development journey. I own all of the Quest headsets (excluding the Quest Pro because it is lame) and I’ve loved them wholeheartedly. But I knew the Apple headset was different. It had to be more than just a VR headset. And it is. It is magic.

And I want in on developing that magic.

Basic App development for the Vision Pro is pretty straightforward. If you want to develop an app that just sits on a flat panel that you interact with, it isn’t much different from developing an iPad app. The code needed for such an app has become pretty well optimized and approachable for the most part. Most iPad apps can run on the Vision Pro, in fact. And that is wonderful. But this device is capable of so much more than being an iPad strapped to your face. We are in the era of spatial computing. That means in your face and all around you content. This is where the ease and intuitive development falls apart a bit. It is very uncharted territory, and it feels more like inventing recipes rather than using established methods. It isn’t easy to learn, not much has had time to become the standard way of doing things. There aren’t libraries to ease the development pains for game development, controlling objects, much of anything. The documentation is all over the place, and nothing seems as cohesive as I’d expect from Apple. This isn’t a bad thing necessarily, just that there is a lot of work to be done. Work that I’m happy to have as part of my journey.

It is possible to develop games for the Vision Pro using Unity. This ability is unfortunately currently kept from the masses behind the paywall of an expensive monthly developer subscription. That’s something I’m not interested in. I find a tremendous amount of allure in learning to develop games natively in the environment designed for it anyway. The problem is, nothing is set up for game development for Vision OS in Xcode yet. In iOS/iPadOS development you at least have SpriteKit to work with. There isn’t anything like that yet though. I’m sure there will be later down the line. Currently they’re pushing TableTopKit geared toward developing board games and such easily. I think that is a good start, I think table top games are a fantastic use case for the Vision Pro, especially with the easy integration of multiplayer. But for traditional games, it is a mess. The physics system isn’t very flexible, the object entity system is cumbersome, it just isn’t a good experience.

So, I’m planning on working on my own game library to hopefully ease the development process for myself and anyone else that may want to use it. I had actual headaches just trying to make a pong prototype function properly. Do you know how many pong prototypes I’ve made over the years? This one almost broke me. I finally got it working, but I want to save others from the same headaches I put myself through. I want there to be a welcoming starting point for people to begin with when learning to develop for VisionOS. I rebuilt the Pong game using the beginnings of my game library, and it was a significantly easier process, so I believe in it and want to make it a reality to help others build their visions for the Vision.

I’ll have a page in the Projects section of the site dedicated to the project and will likely host it on GitHub for anyone to use.