There is prior discussions on this very issue there: Proposal: Make Set covariant
I agree with @alexknvl that the issue is due to the fact that contains currently assumes it can compare anything for equality – which it shouldn’t, even though lack of parametricity on the JVM happens to allow it.
For what it’s worth, I think the multiversal equality proposal for Dotty could fix this, as I have already discussed there: https://github.com/scala/collection-strawman/issues/100#issue-233863714
Of course, a more traditional and rigid Eq[T] type class would also work, but is probably off the table for the foreseeable future of Scala.