DropboxMacUpdate: Making automatic updates on macOS safer and more reliable

Keeping users on the latest version of the Dropbox desktop app is critical. It allows our developers to rapidly innovate, showcase new features to our users, maintain compatibility with server endpoints, and mitigate risk of incompatibilities that may creep in with platform/OS changes.

Our auto-update system, as originally designed, was written as a feature of the desktop client. Basically, as part of regular file syncing, the server can send down an entry in the metadata that says, “Please update to version X with checksum Y.” The client would then download the file, verify the checksum, open the payload, replace the files on disk, restart the app and boom! It would be running version X. This meant that the client had to be running in order to update itself. More importantly, it also meant that small bugs in other parts of the client could affect auto-update. Eliminating these potential failures was crucial to maintain continuity of Dropbox’s value to its users. So we decided it was time to move our auto-update mechanism out of the main app.

Back in 2014, we accomplished this on Windows by taking Google’s Omaha project and adapting it to our needs. Since Omaha is an out-of-process updater, if we shipped a completely broken client we could still update it. This project took a while to finish since Omaha is also an installer/meta-installer and we had to rework several of our installation flows to make it all work well. But we were happy with the end result.

Last year, we decided we wanted to do the same for macOS. Usually we like to start projects like this by doing lots of research. Why reinvent the wheel if you don’t have to? Google did have an open source project called UpdateEngine which was essentially “Omaha for Mac,” but the last code drop was back in 2008 and it wouldn’t compile cleanly with modern XCode, so we decided not to use it. Other options we looked at had other difficulties. Some were in-process only, or supported only one app, or only supported Sierra (we support Dropbox on some pretty old versions of OS X) so we couldn’t use them.

So we decided to write our own auto-update system. This gave us a lot of flexibility in the feature set, and rather than bolting stuff on after the fact, as we did with Omaha, we could build the exact system we needed. (We also didn’t have to use XML for the API 😃.)

• Safety. We must ensure that users only run the code that they should be running.
• Speed. We want to update our users as fast as possible.
• Simplicity. This covers both the design and the implementation. Simple code with lots of unit tests.
• Separation of concerns. The details of how an app is updated should be separate from the method of getting the payload.
• Error handling. Handle all the most common errors correctly, e.g. Network connectivity.
• Logging. Log as much as possible to detect quality degradation.

We built an “app” called DropboxMacUpdate. Because we needed to support old systems (Mac OS 10.7+) we wrote it in ObjC rather than using Swift. Picking one of Apple’s languages let us leverage many of the OS features without too much trouble. (In comparison, the client is written in Python, and when we need to do some OS-specific thing we have to write lots of bridge code.)

Upon installation, DropboxMacUpdate.app will register itself with launchd . This is a well known technique that will allow the app to periodically check for updates. Any Dropbox app can register with DropboxMacUpdate by giving it the path of where it’s installed. Every five hours, DropboxMacUpdate will check its registration database for apps, then check the paths of those apps for the installed version and send them to the server. The server will check if an update is needed and will reply with the version, the URL and the hash of the payload. DropboxMacUpdate then downloads the payload and uses it to update the app. This all happens without the need for any user interaction.

At a minimum, the payload is a DMG with an executable file called .dbx_install at the root. Those of you familiar with Google’s UpdateEngine may see some similarities here. The executable is in charge of doing all the work needed to install the new version of the app. In practice, the DMG will also include the .app that needs to be installed; however, this format allows us to (someday) create a DMG that can update the app using diffs, for example. Notice that DropboxMacUpdate doesn’t have to know the details of how to update the app. This means that if your payload won’t install for some reason, it’s not a problem; just have the server give it a different payload next time (one that actually installs), and you’ll be out of your predicament in no time.

Shipping updates is no trivial matter. We have to ensure that there’s no way for a user to get a “bad” version of the client. So we employ multiple layers of security checks.

1. By default, the connection to the Dropbox server uses TLS with certificate pinning. This way we know we’re talking to dropbox.com and only dropbox.com
2. The server replies with the sha256 hash of the payload. This gets verified once we download the payload.
3. .dbx_install will verify that the .app is signed by Dropbox before copying it to its final location.
4. Additionally, on 10.12+ systems the DMG payload is verified as an in-depth security measure. (Sadly, DMG signature verification is not implemented in earlier versions of OS X.)

But most importantly, being able to deliver secure, reliable, and rapid updates to our users is the biggest security improvement.

Because DropboxMacUpdate can pretty much run at any time, it might try to update a running application. For the Dropbox client we wanted to be able to ask the app to quit so the update could be done as soon as possible. So .dbx_install will find the running app and ask it to exit, using a Darwin n o tification. The client receives the notification, ensures that the app isn’t showing UI, finishes the current sync operations, and exits cleanly. .dbx_install will then swap out the Dropbox.app atomically and restart it. If the client is busy showing UI then we’ll wait for some time before quitting to make sure the user experience is not jarring.

Many of our beta users have been running DropboxMacUpdate for the last months and have benefited from an increased speed in how fast they receive the latest version. In fact, we can update about 3000 clients/sec at peak! We’re excited about shipping this to all our macOS users with version 21 of the desktop client. Happy updating!