Looking at https://spark.apache.org/docs/2.4.0/api/scala/index.html#org.apache.spark.sql.Column it looks like $"column" can be used as an alternative syntax, which to me seems short enough and has the advantage that it doesn’t rely on implicit conversions. Is this the recommended way to make a column ?
In general, replacing the use of symbols in DSLs by string interpolators seems like a win to me, a string interpolator does not require an implicit conversion to add new methods, and if the string interpolator is documented, users should can just hover over the prefix to understand what semantics it has, unlike symbols where the documentation of scala.Symbol won’t tell you what meaning a particular DSL has decided to ascribe to it.
6 Likes
Yes I personally recommend users using interpolation, because that also works for column names with space. However, most users never listen to me 
4 Likes
@rxin do you mean they still prefer to use tick ' ?
In my team, we use something called nextflow, the only way to use it, is to read 100 pages of manual and to remember it by heart. You can not use any construct from the language it is built on (Groovy). (Although you can write groovy and use it, it doesn’t make sense since you have to follow the frameworks design pattern). It takes of course 4 min. to get started, because it is very simple, but when it comes to scaling, you have to read all those pages and still having a hard time to make more complex things work the way you like.
I find that the only thing that counts for these people are syntax. They don’t use language paradigm as we know it. They don’t even use an IDE, that makes it possible to hover over something. They don’t care about the difference between Symbol and String either, and they will not get confused by it.
They didn’t even have syntax highlightning for visual code until recently …
Also, sorry about my bad example yesterday.
For reference:
https://www.nextflow.io/docs/latest/index.html
Yes a lot of users (guestimate > 50%) still prefer to use tick.