Compiling Scala and Sbt for Debian distro

#1

Hi, some months ago I was looking into compiling scala for the Debian project, but got caught in the circular dependency of Sbt/Scala and buld.sbt (among many other related questions such as jdk version compatability, debian packaging rules and so on).

One thing that stuck with me from those discussion, was that @SethTisue and @eed3si9n made som interresting comments about compiling for Debian use.

As far as I remember, one needs Sbt to build Scala and Scala to build Sbt, there is currently no other way to build either. But the build process for Scala and Sbt can be simplified for Debian packaging purposes, as testing and QA of code changes is performed by the respective projects.

The strategy would, I suppose, be to compile Scala by a simplified process, and use that as a basis for compiling Sbt by a simplified process.

What is the simplest way to compile Scala from command line, to make a a release usable as a Debian package and to buld Sbt? Can it be done with Javac, some library dependencies and perhaps som java code generationt tools, from command line? (I assume so, just wanting to check the strategy first, then figure out how to do that)

Regards

Thomas

1 Like
#2

No. You need a previous recent enough version of scalac to compile scalac, we discussed this before: Question about building scala and sbt proper

#3

Yeah, that last discussion pluss the other discussions I had at the same time was a confusing experience…

Put another way, even the scala compiler is written in Scala…

I am not quite ready to give up though, even if its a gordian sized problem. So exploring it from another perspective…

Scala programs can be run from a java command line, with the scala jar file as a classpath dependency. So two questions here:

  1. do you thinkg a basic scala compiler system be built running from java with the Scala jar file
  2. would it be possible to decompile that scala jar fil into java code, to be compiled back to a minimal scala runtime environment, enough to compile a scala release?

Regards

Thomas

#4

I don’t understand this question, what does “running from java with the Scala jar file” mean ?

It’s not impossible, but it would be a lot of work.

#5

I do not think this is really a new problem.

For example, GCC is written in C. You need GCC to build GCC.
Usually one simple download a prebuilt version, and use it to build the version one wants.

I would assume that is what is done to build the debian package for GCC, but it may be worth checking what is their solution.

#6

On top of being a lot of work, I don’t think that this is enough to get your package in Debian, quoting from https://ftp-master.debian.org/REJECT-FAQ.html :

Serious violations (direct rejects even if we only find one point)
…
Generated files: Your package contains generated files (such as compressed .js libraries) without corresponding original form. They’re not considered as the preferred form of modification, so you will either have to provide corresponding original form, or remove them from your tarball, eventually depending on an already available packages to provide missing features.

Decompiling Scala classfiles into Java source files is definitely not going to produce anything that can be called “the preferred form of modification”, even if it’s technically source code.

As I said last time we discussed this, you’ll need an exemption from the normal rules to get an up-to-data scalac in the archive, this is the only sane way to do it. Debian has many other bootstrapped compilers in their archive, which must have gotten in through some process, you just need to figure out what that process is, I suggest asking on the relevant Debian mailing-list.

3 Likes
#7

The problem is its actually impossible to fully comply with GPL licences. Infractions are ignored when they are seen as necessary good for the GPL movement. Underlying the GPL movement is a hidden authoritarianism. This problem partly came to a head with the recent row over Linux litigation. According to GPL system anyone who has contributed a single line of source code to the Kernel can sue any Linux kernel user who is infracting against the GPL2 licence. But just about every major corporate user of Linux in the world can be found to have violated the GPL. Hence the vilification of those that were only doing what the GPL movement has told them to do - enforce the GPL licence conditions.

For starters you can’t write GPL or Linux code to run on an intel or AMD processor. Did you receive the firmware source code with your last CPU? Will Intel send you their firmware source code on request?

#8

Getting a package into Debian has nothing to do with the GPL license, your rant is off-topic here.

3 Likes
#9

Seconded.

This is the answer, this is the only answer, this has always been the answer every time this has come up over the years. There is only one answer to this.

3 Likes
#10

Since Scala 2.11 and sbt 0.13 are in Debian, it would require building Scala 2.12.0 with Scala 2.11 and an old sbt 0.13. This is the loose thread that was missed during the Scala 2.12 rollout, and was probably difficult since it was the first release without the Ant build config.

Unfortunately, it’s a difficult inflection point because there’s a good chance there are a lot of intermediate builds during Scala 2.12 development because of binary breaking changes that were made.

#11

Yep: Question about building scala and sbt proper

#12

The Scala 2.12 compiler is written in Scala 2.12 and uses the Scala 2.12 standard library. Compiling it with 2.11 isn’t an option.

(I’m seconding Guillaume, but I’m trying to put it in the plainest possible terms.)

1 Like
#13

Sorry for my ignorance, but how is the very first Scala 2.12 version (v2.12.0-M1) built, if not with the last 2.11 release?

#14

From my discussions with the Debian team, it means that a package can not contain binary code built from somewhere else. It means all packages must be built from source code that can be built using Debian built tools on debian servers. So an uncompressed .js source file is ok, beccause its the original source. But a generated .js file must be generated from the original source (another example: IDL definition file) on a Debian server, not by an external machine.

So my assumption is that by decompiling scala into java, that source code (as long as the Scala license allows) could be used to build a minamal scala runtime to be included on debian tooling server (dont remember what that server infrastructure was called at the moment) so that it can be used as a legal tool to build a proper scala distribution on the proper Debian build servers. (actually releases: testing, unstable etc, which have their separate build environments and tools available)

#15

I dont remeber the exact history of the GCC development, but I think that the first version of GCC was written by hand by Richard Stallman in 1984. I dont know how it was built. that part is quite scetchy, but several other languages are mentioned, so maybe one of those language environments were free.

#16

And a generated .java file must be generated from the original .scala source, except you’re proposing to upload directly to a Debian server some generated java files, do you see how this is as problematic as generated js files?

#17

I would prefer that solution myself, but the Debian team does not. I discussed this with them the last time and they were adament that the only acceptable solution was to build from scratch using debian tools.

I even suggested simply adding scala to the non-free repository, but they allways shifted the discussion back into how to build it for Debian properly.

The current Scala and Sbt are as @smarter hints, built by the the source package containing a downloaded scala and sbt in a build-lib/ directory. The maintainers did not know this, so they explicitly told me that that was not an option for how to proceed. I think the current Sbt package (0.13) has been, or is going to be, removed from the the proper Debian releases, because of that infraction.

So if this is the only solution, then I ask for help on how to argue the case, by concrete examples and such. Or by figuring out how to get an acceptable Debian build using scala. I know that the tooling server can be used to build early version of tools to use to install on the proper build servers. But I dont know the Debian infrastructure or rules well enough, so any suggestion is valuable.

#18

Just ask them how Rust got into the archive.

2 Likes
#19

I understand what you say, but notice it states “corresponding original form”, which is not the same as “actual original code”

I might be wrong in my assumption, but my argument is the nuance that the IDL file is the source, while the js file isnt. The java file, on the other hand, is a reverse engineered file. Dont know if that argument will pass by the Debian team, but I will ask the Debian team if possible.

My understanding is that they are concerned with the source code being built on Debian servers from scratch and that its is free, not neccesarily that the code has to be “the actual handwritten original source code file”. The source code must be included so its humanly readable and compilable independently of any external tools or dependencies. Thats Debians primary concern as I understand it.

#20

What ? Original means original, there’s no such thing as a “non-actual original”.

Which mean in particular that it’s not the preferred form of modification, which is how https://ftp-master.debian.org/REJECT-FAQ.html defines “generated files”. Without this rule, you could just take any random binary, disassemble it into gas syntax and call the result “source code”. It’s text so it’s technically human-readable, but it would of course never be accepted into Debian.

1 Like