I am trying to build scala and sbt from scratch and I have a couple of questions.
For scala:
On git for scala it says that 2.11 is built using java 6 and 2.12 is built using jdk-8, but on the download pages at scala-lang.org it states both version require java 8. Is it correct that one should use java 6 to build scala 2.11 or run scala 2.11 project code on? And likewise is it correct that 2.12 does require jdk 8 (update #?) and is not completely compatible if run on jdk 9/10/11?
Also is it correct that scala 2.12 should be built with sbt 0.13? I tried compiling it with sbt 1.x but that failed.
Typically it stated:
java.io.IOException: Scala signature package has wrong version
expected: 5.0
found: 5.2 in package.class
The last question I have is that it seems to me that scala requires sbt to build and sbt requires scala to build, so how does one bootstrap that tool chain? I understand that one can install a precompiled version of the one or the other, but the dependency is still circular so it is counter intuitive.
I think you misinterpreted the downloads page. For more complete information on JDK compatibility you should look at JDK Compatibility | Scala Documentation
As for the sbt version you should use, sbt should just select the correct version based on the project/build.properties file. If you don’t edit that file yourself everything should be fine. It’s really not worth trying to upgrade that version yourself for this codebase… Any serious development should probably happen in the 2.13.x (and soon 2.14.x I guess) branch anyway.
Building Scala requires Scala, so you don’t even need sbt in the equation to have a circular dependency.
(If it gives you the heebie-jeebies that you can’t build Scala without using the binaries from a previous Scala, it may help to know that the Scala CI scripts include something called the “stability test” — it verifies that compiling the compiler with the previous binary and compiling it with a freshly built binary produce identical results.)
Hi again, after having done som digging, let me rephrase my question, after a bit of context:
I am trying to upgrade Debians Scala and sbt packages from 2.11.12. The debian process is new to me and strict and complex. I respect Debians political point of view, but it makes things a bit difficult to start with. A Deian package must be built from scatch on debian servers. And I only have scala-2.11 and sbt-ivy available.
Hence the question about correct version of jdk, sbt and scala to use when building different versions moving up to 2.12.8.
I know of the compatability matrix. But there are a couple of issues with it.
1- I tried using default scala on ubuntu/debian, which used 2.11.12 and jdk 10/11, to run a scala script and it crashed. when I downgraded to jdk 8 it ran without problems.
2- Github page for scala, states that 2.11 was targeted at 1.6 and 2.12 at java 8.
Since a jdk has library changes, I found it safest and more stable to run using jdk8. jdk 8 will still be around in the wild for a good many years. and its still a bit more work for scala 2.13 to be complete and scala to be completely jdk-11 compatible.
So my first question is: is this a reasonable basis?
My second question is about building scala. I am using scala-sources-2.12.8.tar.gz (first try was with git clone)
is it enough to just use “dist/mkPack” to build a complete scala release for a debian package or must I go through the bootstrapping procedure and run the stability tests and all, as well
also does it contain all that a release must contain, or are there other repos that needs to be compiled as well?
since I dont have sbt available, is there a way to compile sbt without sbt, using only scala 2.11?
is it correct that I actually only need sbt installed on the build system to compile scala, or must it contain a scala installation as well ro be able to run sbt (before sbt downloads and compiles scala 2.12.7)
can I install sbt 1.2.8 on the build system to compile sbt and scala or should I use 0.13 (again I have read documents that state that certain versions must be used to build sbt at different version)
I think targeting 2.12 is reasonable. Whatever you do for 2.12 should work for 2.13 as well, once 2.13 is finalized. (You, or someone, could think later about whether your efforts could be backported to 2.11.)
I don’t see any need for you to bootstrap or run the stability test. Any merged scala/scala PR has already been through that, it’s part of testing/QA rather than part of the build proper.
Not at the moment.
It would actually be an interesting contribution if someone wants to generate a shell script that requires only scalac to build sbt.
Scala used to be built with ant, so it was probably still possible for these distros to build Scala from source by only depending on the Ant dependency.
Wasn’t 2.12 the first version of Scala that was built with sbt and not ant? Seems like a distro would need to go back to 2.12.0 and try to build with Ant? But even 2.12.0 probably depends on a 2.12 milestone, and not 2.11.
Seems like it would be easier to just start from a jar for sbt-launcher or sbt-extras to build Scala 2.12 from scratch.
In all 2.12.0-Mx builds, both sbt and Ant buildfiles exists. The Ant buildfile was carried over from 2.11. I dont know how usable the later milestone Ant build files are, cause the 2.12.0 build does not seem to have an Ant buildfile. Anyone?
(One is not allowed to use externally downloaded tools/apps/libs/jars, to compile a package in Debian. All debian packages must be built with only what is already available in the current debian repo (main) and the source package alone. I.e it must be able to bootsrtap it self. 30 years ago a few people handwrote the first version of the tools and infrastructure, which has evolved into what is today. Those were the simpler days…
Bootstrapping Scala is basically impossible. You will not be able to get around the fact that to compile Scala 2.12.0, you will need Scala 2.12.0-RC2, to compile Scala 2.12.0-RC2 you will need Scala 2.12.0-RC1-1e81a09, to compile Scala 2.12.0-RC1-1e81a09 you will need Scala 2.12.0-RC1-ceaf419, you can keep going back this way in theory until you get to Scala 1 from 15 years ago, which is written in PiCo, a language which is basically Java-with-pattern-matching and whose source code was never released, so at that point you’ll need to write your own compiler.
I don’t know how Scala 2.11.8 got in Debian, but it’s impossible that it got it in through a full bootstrapping process, they must have cheated, so you’ll have to cheat again to upgrade it.
Note that this isn’t unique to Scala at all, to compile the current version of Rust, you’ll need the previous version of Rust, until you get to the first versions of Rust which were written in OCaml, but to compile OCaml, you’ll need the previous version of Ocaml … Someone did break the cycle through heroic effort by writing an alternative Rust compiler in C++: https://github.com/thepowersgang/mrustc, but I’m not aware of any Scala compiler not written in Scala.
I was thinking of trying that, I found some sh compile command in version 0.4.9 of sbt, so I thought I could look at that and the current build file to see if its possible. The sh command used javac…
I see now that you used the word “generate”, did you mean have some sbt code “record” the build process and the actual commands performed and create a script from it?
You can maybe use it to build Scala 2.12.0-M1, which you can then use to build -M2, then -M3 then -M3-dc9effe, then -M4, … It’s going to take a while to get to 2.12.8.
Yes, that seems right, but once you have 2.12.0 you can skip to 2.12.8.
Since I use sbt, I wasn’t aware before this that Scala 2.11 was available in Debian. Having Scala 2.11 available seems like a toe in here. There won’t be a reason to go back to 2004.
Getting Scala on Debian is a gateway to a lot of distributions, so this has a lot of value for furthering Scala. Granted, the bootstrap-from-source requirement is onerous. Scala language enthusiasts won’t be interested in. That’s what the project is good at, though.
It’s too bad sbt was never packaged in Debian. That’s the bigger hurdle.
Should this discussion be moved to a GitHub issue(s) or the sbt Discourse?
Just an hypothetical question, would it be possible to write a simpler scala compiler / lib in e.g. java, that can be used to bootstrap the scala compiler process? that would be interesting to many people, also for learning about compilers and the as a scala language tutorial. (I would definately do that if it meant learning more about compilers and the inner workings of scala)
Another thought, would it be possible to use the disasembler on the class files, to get pure java source to be able to compile at least a basic scala compiler environment? or was that too naive?
Yes, but it would take a huge amount of work (if the goal is to make a compiler that is only usable for bootstrapping you could cut some corners, but this is probably still one year of work).
You’d probably have to do some substantial changes to the decompiled code to get it to work, so again a lot of work.
Interresting, I suspect thats for one or two reasons. Either the maintainer team did not notice the infraction (which is what happened with the sbt package thats in Debian unstable) or the maintainer team consisted of other people which might have judged it slightly different than the java team. Another possibility is that for ports such as that, they decided to allow it, for some reason. Perhaps otherwise it would never happen or something… Pure speculation though…
Good to know which options are realistic and which are not
I know getting Sbt into Debian is the better choice, but I am exploring alternatives so…
One thought that came to mind was to perhaps use the ant build files from v2.12.0-M5 to build the scala source up to 2.12.8? Would that require much change of the build.xml files and support files? Or put another way, did much change in the way scala is built from M5 and up to 2.12.8?
(I have been looking into alternatives for a couple of weeks, and I keep running into the chicken and egg problem of scala and sbt all the time. So just trying to figure out if there is some solution I can work with in the mean time…)
