Reflections from Apple
May 01, 2024
I’ll never forget the day when I first laid my eyes on the iMac G3 computers in my 7th grade English class. Coming from Nepal, where my most intimate experiences with computers were confined to 30-minute sessions at a neighborhood cyber cafe filled with Intel Pentium 3 PCs, I’d never seen a more beautiful manifestation of a computer before. My fascination with Apple Computer, Inc started here.
During college, when I was studying CS, I always wondered: what would it be like to work at Apple? How did they build and ship high quality software and hardware at such an enormous scale? How did they mysteriously conjure up some of the most iconic computing devices known to mankind? I loved dreaming about the idea of working at Apple, but dreaming wasn’t good enough. I needed it to become reality.
And so it did. But allow me to ruminate here for a bit. Since college, I had fallen in love with Apple platform development. I loved Apple’s SDK, and trying out UI or animation ideas on the iPhone was nothing short of magical. All of my school projects involved iOS apps if I could help it. At Viget (my first job out of college), I championed iOS development during my entire tenure. It was around that time that ARKit was released and taking on the mobile AR world by a storm. VR was taking off, too. I wanted to deperately work in this space, and not just build little side projects.
In 2020, I set my sights on joining Apple, because I realized that no company was really going to be able to deliver mixed reality at scale at the quality we need. But I also needed to practice interviewing, because big tech companies have a high bar for hiring. I interviewed at every single company that would give me a chance. I spent the entire year practicing interview questions and studying algorithms and data structures with the hope that one day I would land somewhere within Apple’s AR/VR epicenter, which was at the time referred to as Technology Development Group (TDG in short). Here’s a snapshot of my interview database:
None of them panned out. I actually failed all but one of my interviews. I had an offer from an early stage startup called Cue, which I ended up taking because it would still allow me to grow and get closer to my goal. It wasn’t the direct path that I wanted, but it was a path in that general direction. I had to remind myself that rejections are not failures, but rather re-directions. One of the greatest lessons I’ve learned in life is that you should be stubborn about your goals, but be flexible about how you get there. I feel that it’s important to share the failures along the way and not just cherry pick the wins when telling the story. Not enough people do this. So many of my heroes failed their way to the top, but not many tell the tales of rejection, redirection, and resilience. The path is rarely linear, and sharing these stories helps others understand that setbacks are a natural part of any journey worth taking.
So I took the job, but kept myself open to interviews. By that point, I had failed 2 of Apple’s onsite interviews and 1 phone screen. None of them were for an org that I wanted, but I was disappointed that I failed. I was never good at the LeetCode stuff, but I persisted regardless, spending a couple hours a day after work to slowly sharpen my skills. My ‘algorithm dojo’, a folder on my computer filled with practice problems kept growing, and I quietly and patiently chipped away. I would spent late hours at the office practicing on the whiteboards. I studied everything I could get my hands on: system design, app architecture, multithreading, async programming, Cocoa design patterns, memory management (ARC and pre-ARC days), Apple framework development… all the way to Objective-C history.
Fast forward 8 months, another TDG recruiter reached out. I passed the phone screen and take-home, but I saw it as just another practice round—I had nothing to lose. But the key here was: losing all hope was freedom. I didn’t care if I got the job or not. By this time, I had so many rejections stacked one after another that outcomes simply didn’t matter to me. I thought it was going to be another auto-reject at the onsite stage like all the other interviews. Alas, life works in funny ways. The moment you emotionally detach yourself from outcomes is the moment things fall onto your lap.
I had a job offer from Apple. From TDG. It was my dream company and dream org, working on a dream product: Apple’s premiere mixed reality headset. I bawled when I read the email. The sense of relief and accomplishment. The joy from seeing all of that studying pay off. It was truly a grind that I never want to repeat again. Regardless, it was a testament to the fact that embracing failure is the greatest skill in life, and I’m proud of myself for persevering. Each one of us have our own paths in life. I always felt lesser when I saw my classmates land their dream jobs in big tech, but that was never the case for me as I had a much more long-winded path.
For the next few weeks, I was ecstatic. On cloud nine. I prepared for my new role as an engineer at Apple. My family, my friends, and my wife (girlfriend at the time) were over the moon for me. They all celebrated me, because they knew how badly I wanted this. It was one of the proudest moments of my life. I worked remotely out of Colorado for a year (due to COVID) – it was rough joining such a big company remotely. But I remember the surreal feeling of creating my very first piece of code at Apple. I couldn’t believe I was here.
At the end of 2021, I relocated to Mountain View so I could fully dive into the action.
Anyway, that’s the story of how I got here. I thought it would be interesting to briefly share the things I worked on to recap my journey on the inside.
On my first team—visionOS System Experience Engineering—I worked on App Intents, which was a brand new framework we built from scratch to expose core app actions to Shortcuts, Siri and Apple Intelligence. Leveraging technologies like XPC, daemons and a myriad of Apple’s internal frameworks for deep system-level integration on visionOS. This practically allows you to look at stuff (like apps, people icons, environment icons) inside of visionOS and say “Siri, open this/message them/FaceTime her”, as demonstrated on this visionOS + Siri guide. From a developer tooling and design standpoint, some of the problems we solved were fascinating, like working with the Swift compiler team to extract (at compile time) type metadata from developer code that adopted our framework’s classes/protocols, all the way to crafting a modern, beautiful meta-API for developers.
We premiered the framework at WWDC 2022, and it’s been inspiring to watch other app developers praise the API for its ergonomics and ease of use. We truly abstracted complex ideas into programmatic prose. It was awesome.
Working on App Intents for visionOS was very cool. I learned about the deep internals of iOS and visionOS, how it’s a complicated symphony of numerous processes communicating to each other for something as simple as launching an app from the home screen.
Eventually, I itched for something different. I desired more product-facing experiences in the realm of UI and graphics. As part of Apple’s Boomerang program, I did a 3-month rotation as a Prototyping Engineer with the Camera Incubation group at Apple Park. Much of what’s worked on there is early research and product exploration so it’s strictly confidential, but it was fascinating to be a part of that group briefly. There were demo days where all sorts of interesting product ideas were pitched to executives and I got to sit in some of those. I got to work on some cool stuff related to lightfields there. At the time, my wife also joined Apple as Process Engineer on iPad displays. It was awesome to commute to Apple Park, get coffee, have lunch and return home together.
After the rotation, I moved over to the visionOS Camera group. The Apple Vision Pro announcement was looming, so I got to work on everything from internal macOS tooling to the Spatial Capture app, which is the primary interface for the 3D camera on the Apple Vision Pro. We solved some very fun, interesting problems in the realm of UI, app architecture and 3D rendering. I loved every bit of it. Around that time, NeRF / Neural Radiance Fields for novel view synthesis was popping off and garnering lots of buzz, so I worked on some prototypes with the Video Computer Vision research group. We built a volumetric neural rendering app that ran fast enough on the headset to be a compelling experience and demo’ed it to leadership. I felt quite out of depth with the theory and math behind the training portion, but working on the rendering aspect using RealityKit and Metal Compute Shaders was a ton of fun.
The Apple Vision Pro announcement was such a proud moment for all of us. A career highlight and an experience that one is lucky to have even once in a lifetime. The energy and excitement was palpable. A cavalcade of thousands of engineers, designers, researchers and scientists, reveling in their collective creation. Super grateful I had such a great team to celebrate with.
The EyeSight display, in particular, was a huge surprise to many (including me). As we ourselves watched the keynote for the first time at SAP Center, the hype and excitement when the display was revealed was audible. I recorded the moment from my perspective for posterity.
I then switched over to the visionOS Photos group full time. Here, I got to work on what is most definitely the coolest thing I’ve worked on in my whole career: the ability for users to create spatial photos from regular 2D photos (could be iPhone or DSLR/Mirrorless/old photo scan). We collaborated across the board with CV and ML teams to extract high-quality depth data from monocular images and synthesize a stereo image pair. It was a monumental cross-functional effort, and I mostly worked on the Photos UI / infra for it.
Showing old family photos to my parents and grandparents, and guiding them to convert those precious photos to spatial, always led to audible gasps. It’s the coolest feeling ever to have someone experience your work in this way, and I couldn’t be prouder of that team. A feature with this level of emotional impact is really more than I could’ve asked for. I think I could’ve worked there happily for the rest of my life, because the photos domain goes so deep and there’s still so much work to be done in the world of spatial computing and photos.
Aaaand that’s where it ends—for now, at least. Life works in funny ways. One day you dream of acquiring something, and another day, you dream of leaving it behind. It’s this very circle that keeps us going; chasing after our freshest desires one after another. The hedonistic treadmill never stops as once you have what you couldn’t dare dream of, you look forward to leaving it behind for something shinier. It’s strange that I’m in that headspace right now… I dreamed of being here for years, and the day to turn my badge in is here.. for now.
It has been a dream of mine since college to take a gap year and spend it travelling, pursuing my own personal projects, making more music/art, taking more photos and videos; sharing all of these artifacts in my own way. It’s time to focus on some of those things. Life’s too short and we can’t afford to spend so much of it working, no matter how cool your job is and how much you love it.
And now that I conclude this piece of writing, the vivid memory of paying 20 rupees for 30 minutes of computer time at the local Internet cafe haunts me yet again, but in a nostalgic way. For 30 minutes, I was computing to my heart’s content. Surely, I will find myself falling in love with that feeling again wherever I end up next. For now, I’m going to stop thinking about computers and go play outside. I do hope I’ll end up back here again, though.
