I really wish there was a good way to systematically request feedback from the FLOSS maintainer base without incurring the bias of who uses discourse, and the relevant people on twitter, and excluding the noise from people who are not maintainers. I’d be willing to answer a few questions every few months, can’t speak for others, but I wonder if that would be something the Scala Center could take on? (i.e. random polls on twitter or here do not count)
Hey Sam,
I think this is an important thing, so I’ve created an independent thread out of it.
I don’t have any ideas on how to get more outreach and get more FLOSS maintainers engaged, especifically to assess the state of the tools / community. But I’m open to ideas, so please let me know if you find a good way.
1 Like
I guess get the github usernames from the github contributor graphs and … create a mailing list 
would need an opt-out and then the amount of questions would need to be minimised so to avoid being annoying.
Yes, I too felt this contributors.scala-lang.org site may be biased due to lack of Spark developers.
1 Like
The wall between Spark and anything else in Scala is still quite amazing to me.
Also, the way this forum always tricks me into responding to threads that have been stale for months, stumbling in after all the guests have left.
2 Likes
It’s ok Som. I feel the same way.
The comments here about the Scala+Spark community caught my eye, and I’m intrigued to know more.
I’m the author of an open source project called RasterFrames, a committer on another called GeoTrellis, have contributed significant PRs to a third called GeoMesa, and all three are major Scala projects built on Spark. I also submit (mostly minor) PRs to lots of other Scala projects, especially in the sbt plugin space, so I don’t see myself as a “Spark-only” developer.
I’m not sure if I count as a “Spark developer” as is outlined here… are you talking about the people who contribute directly to Spark, or those who build libraries on top of it?
I should point out that RasterFrames is intimately tied to the internals of the Spark Catalyst engine, so I’ve been through some of the trials-of-fire that a Spark maintainer might. However, I’ve never submitted a Spark PR… not because I haven’t filed bugs, made suggestions, and even had fixes… it’s more that it’s a monstrously complex project, and the barrier of entry feels more like that of submitting a PR to the Scala compiler… you better damn well know what you’re doing otherwise you’re wasting people’s time.
The processes behind Apache projects feel very opaque to me, but that’s subjective, and I can’t say I’ve yet made the effort to learn more. I’m a member of Eclipse Foundation LocationTech, where the processes–bureaucratic though they may be–are clearly laid out in their public documents. More importantly there’s open discussions on what project roadmaps should be, and the leadership are very open and welcoming and encouraging to suggestions for consideration.
I’ve recently become soured to the Spark project over Scala 2.11 purgatory and another issue around experimental imagery support. I don’t like feeling that way, and really should learn how to engage with that community, but don’t know where to start. Is that an echo of the sentiment here? I don’t mean to distract from @fommil’s original topic, so perhaps this should be forked off, but I’m interested to understand better this group’s perspective.
1 Like
See Spark as a Scala gateway drug and the 2.12 failure for a long discussion on Spark and 2.12, the last post shows that things are hopefully moving along.
1 Like