Active moderation of SIP threads

I think it is very hard restriction. IMHO it will be able to work if free-for-all-thread has been closed already.

May be it has sence to make it softer: one post to sum up per week.

1 Like

This sounds good, so long as the second thread is started after the first one ran its course, this could be a nice improvement.

This doesn’t [sound good], and the way its target is broader than the back and forth discussions which are the main problem would make a negative impact.

IMO

Lots and lots of thumbs up for keeping discussions focused on well-defined topics!

I am not very confident that this is possible, but if it is, it would be splendid. The actual substantive discussions that people wish to have are as voluminous as they are–people have points to make, many of which are important.

But minimizing the extraneous clutter and making it easier to go back and find a conversation would at least allow time to be as well-spent on high-value feedback and discussion as possible!

Again, I’m not sure how to get this to work given how people’s thought processes work. But to the extent that it is possible without massive disruption, I support it, and will do what I can to help!

(Edit–I am doubtful that a hard number limit will do more good than harm. Sometimes issues are complex, and you just cannot get to the heart of an issue in three posts.)

2 Likes

Here’s the situation at hand, from my point of view.

Shared goals:

  • have a discussion

  • hear from everyone who is kind enough and willing to share their experience/expertise

  • hear all sides, challenge our views

  • stay focused

  • stay efficient

  • please keep in mind the fact that the SIP committee needs to follow, summarise, incorporate 40-50 different threads (related to 40-50 feature changes)

  • add time constraint

How we will achieve it:

  • try different strategies, start with 1 structure (for example, as Seth proposed)

  • see how it goes and if it hits all the goals

  • adjust if it doesn’t

  • get it right

We cannot project what will happen until it happens, in these circumstances, but we can promise we will keep an eye on this and be responsive about adjusting.

I will personally be more present as of now in the SIP threads, and more actively participate in making sure that the discussions are fruitful and focused.

And I invite you to mention me or send a private message if you see something going too far and needs my attention, and I haven’t reacted.

I look forward to figuring out this last bit in the Scala 3 marathon!

Thank you in advance!

7 Likes

That’s an interesting idea, and I’m in favor of making a go for it.

My concern is that this process will hinder more complex discussions. Committee members make for a big part in these discussions, so postponing their responses to the end might end the SIP prematurely.

Perhaps there are cases where the SIP deserves more of these cycles, and perhaps not every cycle has to be the same; i.e, first cycles may be much longer than the latter.

I agree with the other commenters that limiting participation in SIP threads to a maximum of three comments per thread would be highly destructive to productive conversation and feedback. I think so long as comments are staying focused and on topic, and are bringing up or responding to points in a way that brings up new information, they are providing valuable input.

That being said, digressions can and probably should be ruthlessly split out to separate thread, and users should try to practice restraint before adding comments which don’t have a reasonable chance of affecting the outcome of an RFC.

IMO we should be if anything trying to find ways to bring more people into the feedback cycle. If the tech platform is holding us back from that, we might consider moving RFC’s to a Github repo + pull requst model, where (a limited form of) threaded conversations are possible. An other advantage of the pull request’s is the wider variety of reactions, so lurkers can provide both positive and negative feedback (GH allows :+1:, :heart:, :-1:, :confused: , etc).

3 Likes

Heavy-handed moderation not only is costly in terms of manpower, but also has a lot of potential for people to feel being treated unfair.

Better, try to give clear guidance to the community so that they can self-organize.

2 Likes

I actually believe that this kind of minimalistic feedback is non-constructive, especially the negative reactions.

I brought this up in the other recent meta-discussion:

This subject was also touched on the talk “The Hard Parts of Open Source” by Evan Czaplicki (which was brought up in the popularity thread).

At 37:45, Evan claims that this type of feedback is a form of de-contextualization, which is helps making topics more viral; then at 38:38, he claims that this is a form of a powerful incentive for our interactions to go really poorly.

He further discusses this topic at 44:02, and explains how this kind of feedback is not contributing to the discussion, and worsening how people feel about their place in the community.

4 Likes

Sorry to sound like a broken record, but I think this quandary is another symptom of trying to do too much at once. If we weren’t trying to fit so many new features and changes into one release it would be possible for feedback to be provided and absorbed much more organically.

4 Likes

Yes, this seems like a much better alternative. Each feature proposal should have a document on some github repo explaining the feature and detailing the pros and cons (possibly with counter pros and cons inside each one), which should be open for pull requests.

The process would stabilize once everyone feels like their point has been properly understood and integrated into the summary of pros and cons.

The back-and-forth will still absolutely be required to arrive there, but it will be absent from the end result (the feature document) from which decisions will be made. This is much better than moving the back-and-forth into other threads that will be promptly forgotten by committee members (who usually won’t have time to follow all of it anyways), which would result in important points and arguments being neglected/overlooked and not mentioned at SIP meetings even more than is the case today.

To me, it’s totally obvious that the current process, and the intense moderation effort that it requires, is not adequate nor very realistic. Even if you manage to make it work with the very small number of participants I have seen in the discussions, maybe at some point more people will join and it will get out of hands again unless an army of full-time moderators is hired to babysit the monolithic discussion.

3 Likes

I believe the proposed “measures” will actually help us (the committee) to understand and integrate more of the feedback. Currently, for the features with the most comments, it has become impossible for us to gather any feedback, because the threads are way too long and without organization (@nafg: this is regardless of the number of proposals; it’s each proposal independently of the others that is unmanageable). By limiting the number of posts per participant, and asking for longer discussions to be taken in separate threads, we can delegate the responsibility of summarizing and structuring the long discussions into synthetic posts (I would personally be in huge favor of disregarding the limit for users bringing back the summary of a side discussion). This way, we would be able to absorb much more of the feedback than we currently can.

@LPTK What you’re suggesting is actually what the ECMAScript committee has been doing since after the publication of ES 2015. I personally like this format a lot, but haven’t been able to convince the rest of committee to move to it yet. :wink:

4 Likes

This is how design documents work in many organizations. You can discuss all you want, but in the end every major point must make it into the document: details, pros, cons, unresolved issues, non-goals, rejected alternatives, migration plans, future work, and so on. Github doc, Google doc, doesn’t matter. Someone will need to curate it, to keep it concise and readable and avoid sprawl: usually the person pushing forward the proposal in the first place.

If you look at the Python PEPs or Java JCPs, they usually at least try to reach a level of comprehensiveness. Someone might not agree with everything inside, but at least they can see what the thought process was without digging through hundreds of emails on a discussion list. e.g. PEP 484 is a good example of a comprehensive proposal for a complex topic https://www.python.org/dev/peps/pep-0484/

I’ve agitated to have the SIP proposals reach that level of quality: so far unsuccessfully, but honestly there is no real choice. This stuff is table stakes for a proper design review. There’s a reason everyone does design reviews this way for any important and complex proposal. It’s a lot of work, but otherwise it is literally impossible to digest the discussion into actionable consensus.

Throwing out a single user-facing documentation page and hoping to find consensus “in discussion” might work for simple improvements, but was honestly never going to work for the kind of complex language changes with such broad impact that we are seeing proposed for Scala 3.

9 Likes

I suggest we move forward with this. It’s a fairly modest change we could do right now, and it wouldn’t require a big increase in moderation labor.

Making this change wouldn’t close the door to perhaps eventually making a bigger change, as some would prefer.

2 Likes

I would just like to add to @lihaoyi excellent points. Its just a fact of life that discussion can never be how the project leaders would like it to be. How much a person gets heard will inevitably reflect the amount of time and energy they are willing to put into debating. Those that are willing to put a lot of time into debate will dominate the discussion.

And I can pretty much you guarantee you that any measures you put in place to hinder their advantage will rebound and have the reverse effect to what was intended. If you limit the discussions to three threads per person, they the really keen protagonist will put three posts in every thread, while the occasional visitor mayl appear for a subject they are interested in be frustrated and annoyed by the 3 post limit and may not bother to come back. If you have two threads per SIP again it will favour those with time and commitment to write everything twice.

If Scala is to grow then the volume of commentary, opinion and debate will inevitably grow as well. And it is the very nature of human discussion that not only do we humans disagree, we disagree about what is being discussed, what ought to be discussed, what should be the limits of discussion, what is relevant and what is orthogonal.

I find meta discussions, separated from the specific sterile. I particularly dislike threads on moderation, which are really about a specific person, people are incidents, where most people are in the know and all talk about the specific matters whole pretending not to and for those who are not in the know, the whole thread makes no sense.

@SethTisue I just want to add that I hope that nothing I say comes across as unappreciative of your efforts and attempts to find innovative solutions. Managing and moderating a programming community is a near impossible task, far harder than most coding problems. I’m very glad that its your job, not mine and am under no illusions that I could do a better one.

4 Likes

I’ve also been thinking about this thread and exchanging thoughts with Seth. We both agree that comprehensive design docs would be ideal in principle, but the reality is that

In the context of Scala 3 features, the person pushing forward is most often @odersky who does not have the time. The SIP committee is working through the individual features, for each feature a champion starts and curates a discussion thread on this forum. We could make the SIP champion responsible for the design doc, but again time is the limiting factor. It is a lot of work, and for noone in the committee this is their job; it’s either goodwill from their employer and/or their free time.

With the pragmatism to

I think Seth’s proposal is actually a very good one. Once a discussion cools down, once the chnages prompted by a discussion made their way back into the dotty implementation and documentation page, a new thread can be started by the SIP champion. The initial post can be more or less extensive depending on how much time is available: it should link to the documentation page, it could summarize the feature and some of the discussions. Then everyone gets one post to make sure their concerns are eternalized. Note that this comment can also be edited, which allows nuancing, re-thinking and adjusting to new developments. But further discussion would have to happen on other threads (the original one, or new ones).

To me this looks like a practicable proposal that’s a step forward and worth a try.

5 Likes

it is very important part of a proposal.
How is it possible to guarantee that changes will be visible to others?
There is checkbox “small changes” in wiki. If the flag is not set there will be a notification with a description.

Is it possible to configure this behavior?

IMHO It is not very good not to be sure that the changes will be visible.

The whole point of the summary thread (as opposed to the open-discussion thread) is to be read once by the committee members after the thread has closed. The idea isn’t to conduct discussion and have further back-and-forth, with edit notifications flying back and forth, as you’re suggesting. The discussion already took place on the discussion thread.

1 Like

We broadly use wiki in our company to organize summary of discussions, it can change
sometimes and notifications allow it to be visible. It does not lead to “back-and-forth” usually and if it happens it does not bloat summaries.

I simply have noticed that ability to edit posts in such themes has very limited value.

Having had some more time to think on this:

Perhaps it’s best if we simply continue as we were before, but with more active attention to moderating the threads. If some thread in particular becomes so long and unwieldy that it’s unreasonable to expect committee members to read it all, then on a case-by-case basis we could adopt my suggestion, by closing the original thread and inviting participants to summarize their thoughts on a followup thread, hopefully yielding something digestible by the committee.

1 Like

:+1: - I really like the idea of being invited to give a summary! Seems like the best of both worlds, with the only downside that if the conversation is mid-stream, the summary cannot address all possible objections and alternate points of view. But that seems like a modest downside compared to the other problems (unwieldy, no follow-up at all, etc.)