The beginnings of Craft on Apple Vision Pro

Craft on Vision Pro - new terrain ⛵️

Why was it important to acquire this platform?

Artom: It’s clear this product is still rolling out, so we're interacting with many early adopters; people who love new tech, or who have already bought into the vision. These customers are valuable because they indicate what future mainstream customers might want.

This allows us to see if we're setting our technology stack up correctly. Mature platforms sometimes limit how you innovate; particularly in how users interact with things. New platforms allow you to break free from what you know, and give you the ability to become a beginner again. When everything feels new and fresh, it pushes us to rediscover what we have learned. Sometimes, this leads to dead ends, and occasionally it leads to breakthroughs. That’s why we're compelled to explore Vision OS early and to get our product out.

Kristóf: The fact is that visionOS is a brand new platform, and does not have a solid set of apps yet. It is the first AR/VR headset not targeted at gamers, but developed with productivity and media apps in mind. As there are not too many note taking apps on the platform yet, we felt Craft could deliver strong value. And we already have a great native text editor, which feels right for visionOS.

Peter: Craft's user base is often tech-savvy, early adopters. Just as Craft is a fresh take on note-taking, they try to keep up to date with the newest tech. We thought that reaching out on a new platform would find more of these types of people.

How did you feel when getting started on it?

Kristóf: We were super excited to work on a brand new platform. Apple did an excellent job of building everything and bringing its whole ecosystem to the Vision Pro, and we wanted to take advantage. Our iPad app is already available to run on visionOS, and we tried the iPad version of Craft on the headset. We felt that it is very good, but that we can do even more if we build a native app for the platform.

Developing the native app was actually easier than expected, Most of our code worked out of the box, and we just needed to adjust minor things. For example, visionOS does not know the concept of "screen", therefore all our code for measuring screen size had to be changed to something sensible. We had to remove a few other things too, like haptic feedback, iPhone status bar color settings, the Mac menu bar app, etc. Then we could make our app run natively.

The next step was to improve it in order to harness the full potential of visionOS. We liked the look and feel of the Vision apps, so adjusted the design of the application to use glass materials, elegant toolbar buttons, and ornaments on the sides of the editor window. We wanted the app to look and feel in place in your room.

Peter: Since its reveal in June 2023, we had kept one eye on the Vision Pro. During our hackathon in October, one contestant aligned their solution to work well for Vision Pro. A new platform always allows new capabilities and directions for a product. Working on a AR/VR platform is not something many of us have done before, so we were excited to find out how to make Craft better in immersive environments.

Turning it on the first time was really magical.

It was good to see how many things were taken care of, such as the setup, and that the selection gesture works straight out of the box without issues.

Craft was already available for Vision Pro as a compatible app, but we couldn't wait to unleash its power natively.

When did you start working on the App?

Kristóf: We treated the visionOS port as a hobby project. Of course, we had other great features which needed work, so we did quick, 1 week-1 person batches of work, starting right after the device went on sale.

How was the design process?

Artom: The process of designing the Vision Pro came in waves. We were lucky to experience the device for the first time, last year, and were impressed by its potential. We were slightly skeptical about whether doing actual work on it would be effective. No other device with VR and AR was ever pitched as a proper device for work. In a way, even the iPad did not start like that.
When we tried the Vision Pro, we fell in love with the device's ability to help users get into a focused state, easily. The design of the OS is so adaptable; it transports you into a totally different world, or totally grounds you in your surroundings. We felt like Vision Pro could enhance our app as a platform, so we invested time in designing and building a proper product experience for the visionOS.

From a design point of view, we noticed that spatial audio was a key enabler for transmitting sensations and giving feedback to users. This felt more natural than on any other device, so we designed a whole series of unique sounds for crucial actions, to help convey the spirit of Craft even more.

On the visual side, we decided not to reinvent the wheel, and utilized as many patterns as possible from the human interface guidelines. We know we can explore many new ideas in the future, but we did not have sufficient time with the platform, or enough feedback from users, to start designing Craft in new directions. But this time will come.

With a radical platform like visionOS, we want to study it first and follow what Apple recommends because they spent many years thinking about the fundamental principles of how humans will interact with the device.

What you will see in our visionOS app is that everything feels familiar to users, particularly if you use the iPad version of Craft. Our iPad app is already one of the best productivity apps in the App Store, but making a native version for visionOS made things much smoother and more accessible.

As for challenges, one of the big ones for design was deciding how far to go with the UX. Which pattern fits the Craft app the most? Should the app have modular parts? Should we show editing options on a glass surface or a plain black one? Should the app shell be dark or light? What should the color of the docs be? All these things needed attention, and we wanted to consider as many things as possible.

New platforms allow you to go further when they are new, and users want to experience something new. So you have permission to try more things and see how they go.

We are already seeing how some of our ideas from visionOS will make their way on to other platforms, which makes the time investment well worth it for the design team.

What were the technical challenges?

Peter: Introducing a new platform to your living and working ecosystem (iOS, macCatalyst) is challenging, mostly because you do not want to lose all the new, shiny features and API, also to ensure that the things are easy to maintain which are exclusively for visionOS, and of course to ensure the app works great without bug fixes.

To achieve this, we separated the project into three main goals:
1. Make it compile
2. Make it maintainable and bug free
3. Make it shiny.

The first part was the hardest. It is one thing to make sure the code under your control can be compiled onto visionOS, but you also need your third-party dependencies to be ready for this. We had to upgrade many dependencies by forking or fixing them.

In the second phase, we made sure our code base was not a forest of ifdefs (we started with 250) and that it blended in with the performant code of Craft which tries to be adaptive and platform-independent as possible. Fortunately, the bugs we encountered on simulators; like invalid placement of the canvas tool bar, were either non-existent on the actual device, or were fixed in later OS versions.

Once we made visionOS a first class citizen in our codebase, we could focus on last-mile efforts to adapt Human Interface Guideline suggestions, and making it stand out from a design perspective.

What were the blockers and some 'a-ha!' moments?

Kristóf: Most of our problems were actually caused by other developer libraries. Since visionOS is so new, not every dependency we use took the chance to update its code yet, in order to "know" the new OS. We had to make a bunch of adjustments, and fork the code here and there to make it work.

I think I fell in love with Craft on the Vision Pro when I first got to see the excellent work of our design team, who created a spectacular glass design for the platform. Craft looks gorgeous with these materials. And it was amazing when we finally got there in the development and could physically "walk around" the Craft window. All the little rounded corners, 3D tooltips, native menus, multi-window handling – these felt very good!

Peter: Fortunately, we could avoid real blockers by making hard decisions about cuts which users might not see, but which made our lives easier, like turning off unsupported features, or third party libraries not ready for visionOS, yet. After that, only hard work remained. Making sure we can generalise the concept of "screens" across all platforms, handling UITraitcollection correctly, fix extensions (like notification, share sheet).

For me, an "a-ha!" moment was when I first saw the native Craft vision Pro app with its glass design, hover/focus effects, and tooltips in place. I can't wait for the final design to be implemented.

How was the collaboration between engineering and the design team?

Kristóf: At Craft, we collaborate very closely between teams. In the design phase, we had several meetings to discuss ideas, and what was possible from a technical standpoint. We reviewed the development of the idea into a design which we implement very frequently. We also let anybody in the office to take home a Vision Pro unit to try it out, in order to have a better idea of what it's capable of. This helped a lot in discovering great features and getting a feel of the design language we wanted to use.

Peter: Separating the project into three steps helped the engineering and design team to work in parallel. While we fixed the code, they could adopt the HIG (Human Interface Guidelines) and play with the currently available apps. Our designers spent hours in Vision Pro to understand its design language. The engineering and design worlds could unite when making it easy to maintain and bug free. While we fixed bugs, they could suggest some smaller refinements before focusing on design in step three, to make it feel native. Designers also collected bugs and improvement ideas that we prioritised and scoped together, before step three.