Scala-json-ast SP Proposal


#21

I thought there was no more SLIP process. SPP is it. Am I misunderstanding? @jvican?


#22

My bad, I was trying to refer to individual libraries being submitted to the SP


#23

It probably doesn’t help that we don’t have any guiding examples of Scala Platform libraries, but from what I understand from this proposal is that it is much anticipated and expected to gain high adoption. The only thing stopping it from gaining high adoption is it not having a stable release, and the only thing stopping it from having a stable release is it not being accepted in the platform (yet).

But you indicate there won’t be a separate platform release.

So what I don’t get is how being part of the platform is going to help producing one, if a stable release is a requirement for being part of the platform - in other words, what a stable release as a platform library is going to accomplish over a stable release as a non-platform library.


#24

Right, this is probably a more polite way of expressing my question. Nothing is stopping anyone from using this right now, yet nobody is. Which once again makes me suspicious.


#25

Library authors have explicitely stated that they will not use the library unless its released as a stable version to be part of the Scala Platform, because one of the main points of the library is interoperability (which you can’t get if you need to release a new dependency).

This was mentioned by @ktoso before

If there is an intention for a library to be part of the Scala Platform and interoperability is one of the libraries main goals of the library (as well as the library being a common dependency for other libraries) then its going to be a lot less painful (and more convincing) for library authors to use the library when its part of SP, because they know they won’t need to update the dependency.


First Scala Platform meeting
#26

I’ve written sjson-new, which provides backend-agnostic codecs. It currently supports Spray, Scala JSON AST, MessagePack, and Int (Murmurhash). For more details, I’ve written three part blog posts:

The Scala JSON AST binding is currently in use for sbt 1.0 components, including Zinc 1.x beta - https://github.com/sbt/zinc/blob/576a5303abbc7b8b0d5564f0e85c86a6f441506d/project/Dependencies.scala#L69
Zinc 1.x beta is already in use in some companies.

For us, the motivation to adopt Scala JSON AST is long-term binary compatibility.


#27

Note that the current Scala Platform Process document says of modules, “Platform modules should stress stability and usability alike, and enjoy widespread use in the Scala community. Modules should be of a nature that aids the goal of the Scala Platform and should have compatible licenses with the rest of the Platform modules.”

It doesn’t define the words “should” and “must” and so on, but normally the words should and recommended refer to things that are not absolutely necessary but count against a proposal or implementation. I think it’s totally fair for the lack of widespread use presently for scala-json-ast to count against it, but I don’t think it’s fair for that to knock it out of consideration without further examination, especially in light of expressed interest.


#28

The issue is, these 2 points can conflict with eachother and in this case they are.


#29

No, the need for a standard library in the Scala Platform isn’t lessened just because there’s a library that is already widely used. The batteries still aren’t included until it’s in.

I don’t think we should just dismiss that the “wide adoption” part is missing. Even if it is a chicken-and-egg situation with respect to this particular library, it nonetheless raises somewhat more concern when something isn’t battle-tested.

But when you really need these kind of batteries, it’s worth examining things that one can reasonably expect to be widely used. The question for the committee is whether it fits the goals of the Scala Platform so well that the lack strict adherence to every single guidelines (they are guidelines!) should be overlooked.

And I agree that in this particular case, adoption is a chicken-and-egg situation. It’s very clear that it is: the library authors who might adopt it have flat out said so, repeatedly.


#30

I am not trying to completely “dismiss” the point, I am just saying its relevance is dependant on context of the SPP.

The current design of the AST, for example, is a slightly altered one which many projects have used (its based off the original json4s/spray JSON design). So although you can claim that it doesn’t have widespread adoption, at the same time its not as if its never been used before. Other points is that its design is very trivial (as opposed to, lets say, a HTTP library or a collections library).

Some of the battle testing has already been done, just under a different name. I also suspect that once there is an indication of its approval, library authors will use the library in the next milestone of the major release, where it will undergo a lot of testing.


#31

My point was that the relevance of widespread adoption was dependent on context! I’m not sure whether we’re agreeing, or whether you’re going one level more meta.

Anyway, I don’t think in practice we particularly disagree. I just think that the discussion should be along the lines of “it is worth it because” rather than “it’s totally irrelevant” w.r.t. adoption.


#32

Yeah I think we pretty much agree and we are just getting more meta, as you said.


First Scala Platform meeting
#33

This library has been officially incubated in the Scala Platform, congratulations! This means the following:

  • Library authors have complete access to Scala Platform’s infrastructure.
    • Automatic release process.
    • Drone integration with caching and several customization features.
    • Official group ids are at maintainers’ disposal. They can release under them if they desire so.
  • Library authors will take part into future decisions regarding Scala Platform Process’s rules.
  • There will be a final vote to accept this proposal into the Scala Platform. This final vote will be done whenever library maintainers feel it’s the right moment to end the incubation.

More information in the official Scala Platform Process.

Incubation period

The incubation period is the perfect moment for gathering developers around your library, creating a community, cleaning up APIs (note that changes in public APIs cause binary incompatibility and are done every year and a half), accepting PRs, creating well-documented issues that people can work on, et cetera.

Next steps:

  • Library maintainers accept Scala Center’s Code of Conduct and use it in their projects.
  • Library maintainers decide the license they will use (they can stay with the same they have).
  • Library maintainers decide whether they endorse C4 or not.
  • Libraries have Gitter channels and pertinent CONTRIBUTION guidelines for people to submit paches/PRs!

Remember that taking decisions on these issues is extremely important for creating a community around the modules – our end goal! You can also participate in the current open debates to abstain from recommending C4 / MPLv2 or changing the major cycle to one year instead of 18 months; your opinion is highly valuable, so please comment.

At the Scala Center, we’re planning to run a series of hackathon in well-known Scala conferences to encourage people to hack on open issues of Scala Platform modules and join their community. Our goal is to boost the success of the Platform and help us get to a point where we can all benefit from a high-quality collection of modules. This is why having CONTRIBUTION guidelines, tips to developers and a getting started guide is important – consider writing them.

Scala JSON AST

In the case of this proposal, there has been a lot of interest to have a published jar available under an official group id. Matthew showed interest in our last meeting to use something along the lines of org.scala-lang.modules. I’m currently figuring out the official name for modules with the Lightbend team, but another suggestion would be org.scala-lang.platform. I’ll let you know the final group id that will be fetchable from Maven.

Regarding infrastructure, I’ll contact library maintainers to make the transition in the next days. Thank you very much for getting involved in this collaborative effort to improve the experience of Scala developers all around the world.


#34

@jvican Can you please provide a response to https://github.com/mdedetrich/scala-json-ast/issues/12

i.e. What GroupId should I be using? Also for the new package, I was thinking of putting it under json.ast instead of scala.json.ast

Thoughts?

Also I want to try and get this released before SBT 1.0.0 is released


#35

Hey Matthew, thanks for the ping.

I’ve just answered your question in the issue.


#36

Would like to announce that the first release of scalajson on the proper organization and package name has occurred, you can view the details in the github repo here https://github.com/mdedetrich/scalajson (tl;dr the dependency is now "org.scala-lang.platform" %% "scalajson" % "1.0.0-M1" and "org.scala-lang.platform" %%% "scalajson" % "1.0.0-M1" for scala.js)

Thanks for everyone that has helped


#37

Artifacts are already in Maven Central: https://search.maven.org/#search|ga|1|org.scala-lang.platform.


#38

I wrote down a few notes about how spray-json could integrate with scalajson here: https://github.com/spray/spray-json/issues/232. I think it would be good if the scalajson project itself would give a few suggestions how to integrate.

I guess, so far, the simple “shallow” integration is the most likely outcome for spray-json.

I wonder if it is too late to make any substantial changes to the proposal? My suggestions would be removing the unsafe AST and not providing a concrete implementation of the AST at all but only interfaces of the AST nodes for json libraries to implement. What do other maintainers of other json libraries say? (See the discussion for circe here: https://github.com/circe/circe/issues/690#issuecomment-311956866) Is this a forum where json library maintainers could discuss these things?


#39

I will respond in the issue on spray-json


#40

As a maintainer of play-json I pretty much agree with @jrudolph. I think the goal here should be to provide common APIs. It’s great to have another set of models for helping with parsing, but I don’t see why that needs to be part of the Scala platform, especially since I don’t think everyone will agree that those specific models are the right one to use. For example I don’t agree that Array/js.Array is necessarily the most efficient data structure to use for JSON objects.

The AST for the “safe” version is very similar to the play-json AST so we could probably easily do the conversion in play-json 3.0.