Swift Adds Ownership (Kind Of)
I extol the virtues of Swift's ARC memory management only to discover that Apple has added Rust-like ownership to Swift.
Materialized View blew past 2,000 subscribers in April! I’ve received a lot of positive feedback and I am humbled by its growth. Thanks!
I’ve been hacking with Rust lately while building a cloud native LSM database. The learning curve has been steep and Rust’s borrow checker has gotten in the way a few times. Over breakfast a few months ago, one of my (Apple-fan) friends lamented that Rust should have adopted Swift’s memory management if, “infrastructure developers could just get over themselves.”
Swift uses automatic reference counting (ARC) to manage application memory. For those unfamiliar, ARC counts pointers to data on the heap. When there are no more pointers to data on the heap, the memory is freed. This stands in contrast to garbage collected languages like Java and Go, or manual management like C, C++, or Zig.
There are, of course, rough edges to ARC. You can create cyclic references that never get freed, for example. Still, it’s (usually) fast and works pretty well. I posted this thought, and got a really interesting conversation.
I did some digging, and Swift’s evolution on the subject is interesting . It turns out that someone from Apple posted an Ownership Manifesto proposing a more Rust-like ownership model more than 7 years ago. The problem statement section largely summarizes the discussion above:
The widespread use of copy-on-write value types in Swift has generally been a success. It does, however, come with some drawbacks:
Reference counting and uniqueness testing do impose some overhead.
Reference counting provides deterministic performance in most cases, but that performance can still be complex to analyze and predict.
The ability to copy a value at any time and thus "escape" it forces the underlying buffers to generally be heap-allocated. Stack allocation is far more efficient but does require some ability to prevent, or at least recognize, attempts to escape the value.
Since the original post, it’s gone largely untouched. Or so I thought. It turns out that there’s been more than three years of discussion on a Swift forum post called A roadmap for improving Swift performance predictability: ARC improvements and ownership control. It’s a long and detailed history of the work going into Swift’s memory management. Work that has now surfaced in in Swift 5.9, released in September of 2023.
Swift 5.9 adds a bevy of ownership-related features:
And—wait for it—borrowing and consuming parameter ownership modifiers
The release notes have an ownership section describing the changes:
The new `consume` operator tells Swift to deinitialize a variable and transfer its contents without copying it. The `consuming` and `borrowing` parameter modifiers provide hints that Swift can use to eliminate unnecessary copying and reference-counting operations when passing a parameter. Finally, noncopyable structs and enums allow you to create types which, like a class, can’t be meaningfully copied when assigned, but like a struct or enum, do not need to be reference-counted because only one storage location can own the instance at a time.
There’s also some discussion on HackerNews, including this spicy comment, “For current Rust users, if Rust kicks the bucket, Swift is a good plan B.”
I’m torn on these changes. On the one hand, I like Swift’s relatively simple memory management. On the other hand, I buy the argument that ARC can be slow in certain circumstances. Some of ARC’s edge cases happen to matter a lot when building deep infrastructure—the kind of stuff you build with Rust. Perhaps Swift has managed to thread the needle with their ownership semantics in a way that Rust hasn’t. If not, there’s always Mojo.
Jobs
Modal [$] - Modal is one of the coolest kids in the AI space (and in the serverless infra space. The team has been growing and they’re now looking for AI, product, and systems engineers. They work with open source projects like gVisor and have home-grown tech built on Python and Rust. Apply here if this is your thing.
StarTree [$] - One of my former LinkedIn co-workers, Kishore Gopalakrishna, recently posted a mysterious recruiting request. The team is looking for, “humble and hungry engineers who want to work on a really exciting but very hard project.” StarTree has productized Apache Pinot, a slick realtime OLAP system that leverages star-tree indexes. Oh, and StarTree’s founding engineer is none other than Chinmay Soman, one of the first engineers to work with me on Apache Samza. Send Kishore a DM to work with them (they’re great).
BlueSky - I am continually amazed at the craftsmanship coming out of BlueSky. I’m also rooting for them as a company. While reading Jaz’s recent Your Data Fits in Memory (GraphD Part 1), I noticed they’re hiring backend engineers. They’ve got a Go backend, though Jaz is messing a lot with Rust. They’ve also got an intriguing architecture (including *gasp* on-prem hardware). They’re having a lot of fun and building great stuff.
Book
Support this newsletter by purchasing The Missing README: A Guide for the New Software Engineer for yourself or gifting it to someone.
Disclaimer
I occasionally invest in infrastructure startups. Companies that I’ve invested in are marked with a [$] in this newsletter. See my LinkedIn profile for a complete list.