- The Typelevel stack now offers a complete platform for Scala Native: concurrent fiber runtime, non-blocking network I/O, TLS, HTTP server+client, Redis client. Coming soon: Postgres client, serverless integration, server/client code-generators. It is now more viable than ever to build production applications on Scala Native.
- The success of (1) is thanks to years of hard work and demonstrates exactly how far Scala Native has come. But the project is also facing some immediate challenges, such as maintaining Scala 3 support, that may threaten adoption.
- Scala Native has a vibrant community working on core, tooling, and libraries. Wojciech Mazur is beyond-words phenomenal, but is the sole maintainer with only a “fraction” of their time available for the project. The current commitment of Virtus Lab and Scala Center to the project is unclear to me.
- What additional resources can be dedicated to the project, or could be made available in the future?
- Specifically, what milestones would stakeholders like to hit to encourage allocation of said resources?
- What are the technical next steps for the project to best achieve those milestones?
At the bottom of this post I have included a list of some Typelevel stack libraries that recently cross-compiled for Native or I anticipate will do so soon. I have selected this subset of libraries because I believe they are novel offering on the Scala Native platform that (1) play to its strengths and (2) lower the barrier to adoption.
These libraries are largely designed for building concurrent services with asynchronous, non-blocking I/O. They are not trifled by Scala Native’s single-thread limitation and enable you to serve 1000s of connections simultaneously. This is effectively the Node.js model.
Furthermore, thanks to native interop, the runtime can be deeply integrated with kernel I/O APIs such as epoll, kqueue, and io_uring. Indeed, the epollcat project mentioned below implements one such runtime. Combined with instant startup of native binaries, I believe there is real potential for an interesting, competitive offering for deploying scalable, I/O-bound services with Scala Native.
With the exception of FS2, which implements non-blocking network I/O and TLS, the libraries listed below cross-compile for Native with virtually no changes to their source code. These are not “Scala Native” libraries, these are bona fide Scala libraries that cross-build for Native with identical implementations, APIs, and semantics to their JVM counterparts. Downstream libraries and even existing applications can cross-build effectively “for free”. I am hopeful this lowers the barrier for adoption.
On the topic of lowering barriers, I think two UX improvements could go a long way towards improving user adoption of Scala Native, for developing applications:
- Incremental linking, so each tweak/test iteration doesn’t take several minutes
- Line numbers in stack traces
I also touched on my concerns due to Scala Native’s unfortunate entanglement with Scala 3 compatibility. As an OSS maintainer, my biggest fear is that due to these sorts of complications I will have to rip Scala Native support out of a library to avoid compromising JVM development.
These problems are solvable, so long as someone is available to work on them. Which is why I am eager to hear from Virtus Lab and Scala Center about what their goals for the project are and how we can build on the current momentum.
There was some recent discussion in the old “Scala Native Next Steps” thread (when that thread started there were 1.5 engineers on the project!) of which only EPFL’s commitment is clear, at least to me.
Note this is only a very select subset and feature highlight of what’s available on Scala Native I hope that the new Typelevel offerings are interesting and I look forward to connecting with you in those projects’ channels!
- lightweight fiber (aka green threads) runtime
- userland-extensible event-loop execution context and scheduler
- non-blocking TCP sockets via epollcat
- TLS support via s2n-tls
- Ember, a 100% pure Scala server+client with HTTP/2 support
- wealth of middlewares and ecosystem of integrations
- 100% pure Scala Redis client
In addition, I expect the following libraries to be published for Scala Native in the near future:
- 100% pure Scala Postgres client (save some crypto routines)
- Scala Native PR is already up and CI is green. That test suite is no joke!
- server/client code generators from Smithy IDL
- severless framework for AWS Lambda
… and many many others