On this forum, each project supported through this investment has its own dedicated thread.
This thread covers Scoverage and will be used to share the project overview, a roadmap with key milestones, ongoing progress updates, and opportunities to engage—so we can hear ideas from the community and encourage contributions.
Scoverage is a Scala feature that allows to measure the test coverage in scala codebases. Currently, Scoverage is not tested on the compiler test suite, which means that sometimes, there are changes to the compiler that break scoverage.
This thread is to announce a project under the Sovereign Tech Agency umbrella the goal of which is to harden Scoverage by enabling it in the Scala 3 compiler CI pipeline on the existing suite of compiler tests. By running Scoverage on the existing tests, we will ensure that any new features or changes made to the compiler do not break Scoverage.
Once implemented, the impact of this work is to increase reliability and adoption of Scoverage, making it feasible for more industry players to use in production.
The scope of the work:
Run Scoverage against the Scala compiler test suite in CI
Fix the failures this uncovers
Follow this thread for updates on the work. Also, if you are a Scoverage user, your input on the experiences you are having with this feature is more than welcome!
A semi-tangent, but the fact that $COVERAGE-OFF$ / $COVERAGE-ON$ pragmas don’t work with Scala 3 is a huge problem for us – I only discovered it recently, and it’s a major speedbump in our Scala 3 adoption, since we have a very deeply-set rule that coverage should always be set to 100%.
Don’t know if there is any chance of that changing, but fixing this would make it easier for any serious codebase to manage its coverage properly.
I use Scala 3 + sbt-scoverage in all my projects, it’s very important to me.
In my current cross-project (Scala 3 jvm/js) I get a crash when trying to run the js tests with coverage. According to GitHub - scoverage/sbt-scoverage: sbt plugin for scoverage, “ScalaJS and Scala Native support is limited to Scala 2.”
My ScalaJS tests include statements like org.scalajs.dom.document.getElementById(‘some-id’) which seems to be a problem.
To run these tests in the first place, I needed to add libraryDependencies += "org.scala-js" %% "scalajs-env-jsdom-nodejs" % "1.1.0" to my plugins.sbt file - though after that they worked fine.
But as mentioned, trying to run them with coverage I get an error along the lines of [error] (js / Compile / compileIncremental) dotty.tools.sjs.ir.InvalidIRException: Cannot hash a transient IR node (its value is of class class dotty.tools.backend.sjs.JSCodeGen$UndefinedParam$)
I’m not sure if this the correct place for this specific problems, but this is how I currently use scoverage and my struggle with it.
What Scala version are you using? I think there should be a fix for Scala JS in newest 3.8.x and coming to 3.3.8 (already possible to check via nightlies).