A meta-subject that came up at the SIP retreat this week was that the volume of commentary on the official SIP feedback threads was too high and there were too many unrelated or only-semi-related posts and too much back and forth between individuals. It’s overwhelming for the committee and it’s not really practical to continue this way.
As a consequence, we intend to be much more active about moderating the threads from now on.
Participants are encouraged to start new topics themselves to begin with, rather than waiting for moderators to do it, when something is tangential. It’s fine to post a very short message to the parent thread directing interested people to the sub-discussion.
We will surely make some mistakes about whether something is sufficiently related or not. We’ll do our best.
I appreciate the attempt, but I believe this is not the way to go.
Many of these sub-discussions really don’t merit individual threads. If members really take this advice, we would end up with dozens of threads in the forum, which will make it hard to follow on the major threads being discussed. Perhaps a new category for sub-discussions is in due.
The other problem, I believe, is that once a discussion is “set aside”, former participates will be discouraged to continue the discussion. This goes especially for the participants who are already in agreement with the general notions of the main discussion, as they can now ignore the side-discussion and let the counter arguments die out more easily.
One possible rule that we discussed, and that we may still put into place, would be limiting the total number of posts by any single participant in a SIP thread to three, total.
It would be up to participants to engage in back-and-forth in other topics, and then if they want to use one of their three posts to summarize (and link to) that back-and-forth, that would be fine. The goal is for the threads to contain closing statements, not extended back-and-forth.
I acknowledge the disadvantages you raise, but it is simply not possible for the committee to continue to digest the volume of discussion that is going on. So we need to find a way forward and any way forward is going to have disadvantages.
If the objective is to compile a summary of arguments and points and de-emphasize the back-and-forth, maybe google docs would be a better fit? It could be made world-editable, and discussions could happen in comments.
Alternatively it could be done on a github PR with inline discussions that can each get marked resolved.
Either way the “end product” is a file documenting all the points in favor and against both the proposal as a whole, as well as any aspect of it. It seems like that’s what you really need, since as you say the committee is being overwhelmed by the job of inferring this from free-form threads. By using a tool that makes this font-end-center, and allows it to be updated in conjunction with discussion, I think you will be helping the community in the right direction much more effectively.
If members really take this advice, we would end up with dozens of threads in the forum
Hmm. Perhaps each feature should have one free-for-all-thread, and then one thread where each person is allowed one post to sum up what they think is most important to say? Then the committee would read the latter. (Some committee members might dig deeper, of course.)
(Naftoli’s suggestion is similar in some respects, but I think we are very reluctant to add more technologies into the mix, and also, discussions in Google Docs comments between more than a handful of people can easily become unwieldy.)
This sounds good, so long as the second thread is started after the first one ran its course, this could be a nice improvement.
This doesn’t [sound good], and the way its target is broader than the back and forth discussions which are the main problem would make a negative impact.
Lots and lots of thumbs up for keeping discussions focused on well-defined topics!
I am not very confident that this is possible, but if it is, it would be splendid. The actual substantive discussions that people wish to have are as voluminous as they are–people have points to make, many of which are important.
But minimizing the extraneous clutter and making it easier to go back and find a conversation would at least allow time to be as well-spent on high-value feedback and discussion as possible!
Again, I’m not sure how to get this to work given how people’s thought processes work. But to the extent that it is possible without massive disruption, I support it, and will do what I can to help!
(Edit–I am doubtful that a hard number limit will do more good than harm. Sometimes issues are complex, and you just cannot get to the heart of an issue in three posts.)
Here’s the situation at hand, from my point of view.
Shared goals:
have a discussion
hear from everyone who is kind enough and willing to share their experience/expertise
hear all sides, challenge our views
stay focused
stay efficient
please keep in mind the fact that the SIP committee needs to follow, summarise, incorporate 40-50 different threads (related to 40-50 feature changes)
add time constraint
How we will achieve it:
try different strategies, start with 1 structure (for example, as Seth proposed)
see how it goes and if it hits all the goals
adjust if it doesn’t
get it right
We cannot project what will happen until it happens, in these circumstances, but we can promise we will keep an eye on this and be responsive about adjusting.
I will personally be more present as of now in the SIP threads, and more actively participate in making sure that the discussions are fruitful and focused.
And I invite you to mention me or send a private message if you see something going too far and needs my attention, and I haven’t reacted.
I look forward to figuring out this last bit in the Scala 3 marathon!
That’s an interesting idea, and I’m in favor of making a go for it.
My concern is that this process will hinder more complex discussions. Committee members make for a big part in these discussions, so postponing their responses to the end might end the SIP prematurely.
Perhaps there are cases where the SIP deserves more of these cycles, and perhaps not every cycle has to be the same; i.e, first cycles may be much longer than the latter.
I agree with the other commenters that limiting participation in SIP threads to a maximum of three comments per thread would be highly destructive to productive conversation and feedback. I think so long as comments are staying focused and on topic, and are bringing up or responding to points in a way that brings up new information, they are providing valuable input.
That being said, digressions can and probably should be ruthlessly split out to separate thread, and users should try to practice restraint before adding comments which don’t have a reasonable chance of affecting the outcome of an RFC.
IMO we should be if anything trying to find ways to bring more people into the feedback cycle. If the tech platform is holding us back from that, we might consider moving RFC’s to a Github repo + pull requst model, where (a limited form of) threaded conversations are possible. An other advantage of the pull request’s is the wider variety of reactions, so lurkers can provide both positive and negative feedback (GH allows , , , , etc).
At 37:45, Evan claims that this type of feedback is a form of de-contextualization, which is helps making topics more viral; then at 38:38, he claims that this is a form of a powerful incentive for our interactions to go really poorly.
He further discusses this topic at 44:02, and explains how this kind of feedback is not contributing to the discussion, and worsening how people feel about their place in the community.
Sorry to sound like a broken record, but I think this quandary is another symptom of trying to do too much at once. If we weren’t trying to fit so many new features and changes into one release it would be possible for feedback to be provided and absorbed much more organically.
Yes, this seems like a much better alternative. Each feature proposal should have a document on some github repo explaining the feature and detailing the pros and cons (possibly with counter pros and cons inside each one), which should be open for pull requests.
The process would stabilize once everyone feels like their point has been properly understood and integrated into the summary of pros and cons.
The back-and-forth will still absolutely be required to arrive there, but it will be absent from the end result (the feature document) from which decisions will be made. This is much better than moving the back-and-forth into other threads that will be promptly forgotten by committee members (who usually won’t have time to follow all of it anyways), which would result in important points and arguments being neglected/overlooked and not mentioned at SIP meetings even more than is the case today.
To me, it’s totally obvious that the current process, and the intense moderation effort that it requires, is not adequate nor very realistic. Even if you manage to make it work with the very small number of participants I have seen in the discussions, maybe at some point more people will join and it will get out of hands again unless an army of full-time moderators is hired to babysit the monolithic discussion.
I believe the proposed “measures” will actually help us (the committee) to understand and integrate more of the feedback. Currently, for the features with the most comments, it has become impossible for us to gather any feedback, because the threads are way too long and without organization (@nafg: this is regardless of the number of proposals; it’s each proposal independently of the others that is unmanageable). By limiting the number of posts per participant, and asking for longer discussions to be taken in separate threads, we can delegate the responsibility of summarizing and structuring the long discussions into synthetic posts (I would personally be in huge favor of disregarding the limit for users bringing back the summary of a side discussion). This way, we would be able to absorb much more of the feedback than we currently can.
@LPTK What you’re suggesting is actually what the ECMAScript committee has been doing since after the publication of ES 2015. I personally like this format a lot, but haven’t been able to convince the rest of committee to move to it yet.
This is how design documents work in many organizations. You can discuss all you want, but in the end every major point must make it into the document: details, pros, cons, unresolved issues, non-goals, rejected alternatives, migration plans, future work, and so on. Github doc, Google doc, doesn’t matter. Someone will need to curate it, to keep it concise and readable and avoid sprawl: usually the person pushing forward the proposal in the first place.
If you look at the Python PEPs or Java JCPs, they usually at least try to reach a level of comprehensiveness. Someone might not agree with everything inside, but at least they can see what the thought process was without digging through hundreds of emails on a discussion list. e.g. PEP 484 is a good example of a comprehensive proposal for a complex topic PEP 484 – Type Hints | peps.python.org
I’ve agitated to have the SIP proposals reach that level of quality: so far unsuccessfully, but honestly there is no real choice. This stuff is table stakes for a proper design review. There’s a reason everyone does design reviews this way for any important and complex proposal. It’s a lot of work, but otherwise it is literally impossible to digest the discussion into actionable consensus.
Throwing out a single user-facing documentation page and hoping to find consensus “in discussion” might work for simple improvements, but was honestly never going to work for the kind of complex language changes with such broad impact that we are seeing proposed for Scala 3.
I would just like to add to @lihaoyi excellent points. Its just a fact of life that discussion can never be how the project leaders would like it to be. How much a person gets heard will inevitably reflect the amount of time and energy they are willing to put into debating. Those that are willing to put a lot of time into debate will dominate the discussion.
And I can pretty much you guarantee you that any measures you put in place to hinder their advantage will rebound and have the reverse effect to what was intended. If you limit the discussions to three threads per person, they the really keen protagonist will put three posts in every thread, while the occasional visitor mayl appear for a subject they are interested in be frustrated and annoyed by the 3 post limit and may not bother to come back. If you have two threads per SIP again it will favour those with time and commitment to write everything twice.
If Scala is to grow then the volume of commentary, opinion and debate will inevitably grow as well. And it is the very nature of human discussion that not only do we humans disagree, we disagree about what is being discussed, what ought to be discussed, what should be the limits of discussion, what is relevant and what is orthogonal.
I find meta discussions, separated from the specific sterile. I particularly dislike threads on moderation, which are really about a specific person, people are incidents, where most people are in the know and all talk about the specific matters whole pretending not to and for those who are not in the know, the whole thread makes no sense.
@SethTisue I just want to add that I hope that nothing I say comes across as unappreciative of your efforts and attempts to find innovative solutions. Managing and moderating a programming community is a near impossible task, far harder than most coding problems. I’m very glad that its your job, not mine and am under no illusions that I could do a better one.
I’ve also been thinking about this thread and exchanging thoughts with Seth. We both agree that comprehensive design docs would be ideal in principle, but the reality is that
In the context of Scala 3 features, the person pushing forward is most often @odersky who does not have the time. The SIP committee is working through the individual features, for each feature a champion starts and curates a discussion thread on this forum. We could make the SIP champion responsible for the design doc, but again time is the limiting factor. It is a lot of work, and for noone in the committee this is their job; it’s either goodwill from their employer and/or their free time.
With the pragmatism to
I think Seth’s proposal is actually a very good one. Once a discussion cools down, once the chnages prompted by a discussion made their way back into the dotty implementation and documentation page, a new thread can be started by the SIP champion. The initial post can be more or less extensive depending on how much time is available: it should link to the documentation page, it could summarize the feature and some of the discussions. Then everyone gets one post to make sure their concerns are eternalized. Note that this comment can also be edited, which allows nuancing, re-thinking and adjusting to new developments. But further discussion would have to happen on other threads (the original one, or new ones).
To me this looks like a practicable proposal that’s a step forward and worth a try.
, 
, 
, 
, etc).