Scala Native Next Steps

[jrudolph]

Thanks for the replies and points of view, @rcano and @olafurpg.

Regarding competition: I’m also not a fan of overly competitive thinking when it comes to languages. From an idealistic viewpoint, I would like to see Scala Native to succeed and I could just be happy that there’s funding for a somewhat risky project.

That said, I am also realistic enough to know that resources are limited. Resources are Scala Center’s developer resources, but also public attention and momentum that are required to drive such a project.

Ultimately, the only way (for anything) to survive is find its (probably temporary) niche. The statement at the top about why Scala Native is relevant and what is missing and will be done is too fuzzy to be useful. You could say this is a classic project management problem. If you want such a project to succeed you will first have to admit there’s an insurmountable amount of work ahead of you, and then you will need to have a very specific idea how to arrive incrementally at something useful as quickly as possible. The statement gives a reasonable roadmap for the short-term but doesn’t say what reasonable short- and mid-term goals are and how to reach them.

My assumption behind this is that Scala Native currently isn’t useful (enough). Here again the question is whether you look at “being useful” as an isolated absolute concept or relative to what’s going on elsewhere. Let’s take the stance, that it is useful if, given a systems programming task, you would choose Scala Native over anything else. Let’s look at a few scenarios:

• A simple short-running command line application (dealing mostly with file IO and nothing else, no): yes, Scala Native is applicable even now, you can program in Scala, basic APIs are there
• A short-running command line application that is CPU intensive: not so much, since there’s no support for multi-threading to spread the work
• A tool that connects to other service over the network, not so much, since protocol implementations are missing
• A low-level tool that interfaces with the kernel (basically standard C usage) or C libraries: somewhat applicable and performant but inherits all the bad parts from C (memory unsafety, clunky APIs, etc.) while even missing concurrency primitives for the Scala parts. If you compare that with just doing the same using Scala on the JVM using JNA as FFI, you don’t win much. Using JNA is somewhat uncommon but it works on the JVM if you don’t have the utmost performance or resource requirements. With native-image, you can also solve these with SubstrateVM’s FFI.
• A bigger, long-running application: plus points for being able to write in Scala, minus points for missing multi-threading, missing connectivity, Scala on the JVM already provides all these features with good to perfect performance characteristics (but the well-known bloat wrt startup times and memory usage)

Which use cases are missing and which of these use cases are the primary target for Scala Native?

For me personally, the current answer whether to use Scala Native is “no” (aside from playing around) because there’s nothing useful enough I can do with it. I have a project for quick Linux memory analysis (doing some custom aggregation) that would come closest. That currently runs a few seconds per invocation. But even there, most of the runtime is because its functionality (parsing files in /proc) is not optimized to be fast. Despite it being a short-running program, the JVM imposed runtime cost is there but not relevant enough to try hard to get rid of it.

So, I guess what I’m asking for are more concrete ideas about applicability and then also some data that would support those ideas.

@olafurpg

Scala Native itself doesn’t need built-in support for TLS to be useful in server development, it’s possible to build on top of existing servers like NGINX Unit

There’s not only servers… And also, just having a C-API doesn’t make it a ready-to-use library for Scala Native.

@olafurpg

If we are going to pick any language where Scala Native is “competing” (and I don’t think such discussions are fruitful anyways as covered by @rcano), I would rather pick Python instead of Go or Rust. Having worked full-time with Python, I think there’s lot of applications that are using Python today where Scala would be better suited.

If you are coming from Python you could also use Scala on the JVM, couldn’t you?

@olafurpg

which requires a decent upfront investment to learn. In comparison, Scala Native binaries link significantly faster and you can test them just like you would test a JVM application giving me better confidence that the binary works as expected at runtime.

Yeah, native-image times are really bad (but that’s an ongoing struggle for any compiled language). Would be interesting to see some apple-to-apple comparisons for that particular aspect (but, of course, it’s also somewhat expected that Scala → native translation can be faster than going via bytecode first, the question is whether the downsides are worth it).