Syntax highlighting inside custom string interpolators

Language Design
21

With IntelliJ IDE you can use language injection, which requires a comment, but nothing drastic:

val button = // language=HTML
    html"""<button id="btn" onClick=${handleOnClick}>
      Click Here
    </button>"""

I suppose one can imagine driving language injection not only by comments, but also by context (be it interpolator, function parameter, whatever).

22

I’ll try to give it a look, thanks !

Overall this features seems a lot more feasible than I first thought !

23

For me extra methods are useless, unless we implement them to return something usefull JsonValue YamlValue Seq[CsvRow].

1 Like
24

I’m starting to look into it, here are my findings so far:

  • You can debug syntax highlighting with the scope inspector
  • When using it on a piece of js embedded in markdown, I correctly get language javascript (and markdown when outside of the code block)
  • In scala, an interpolation is the textmate scope string.quoted.triple.interpolated.scala, including the start and end """ (except escaped characters ?) see this
  • Embedded Programming Languages | Visual Studio Code Extension API Seems to be talking about exactly what we want
    • In particular the section about request forwarding
  • The vs-code-scala-syntax repo seems to only contain a json schema and a Scala tm language (whatever that means)
    • In particular, it does not seem to include javascript/typescript code capable of making calls such as the ones needed for request forwarding, should this feature go in Metals instead ?
  • In the conclusion it is mentioned that “Overall, we recommend building a language server by embedding language services [the opposite of request forwarding], as […] the server is reusable for any LSP-compliant editors.”
    • This seems to hint at the fact a request forwarding -based LSP would not work outside of VSCode, in particular might not in Github

Overall I am now very convinved this is not only possible, but achievable in the short term

@tgodzik you were the one to point me in the direction of vscode-scala-syntax, given the above, how do you think we should proceed ?

25

That one is more complex, it includes things like auto completions etc. so full language server features. It is a much bigger task than just syntax highlighting and might actually not be possible to do syntax highlighting using this way as not all language server might provide semantic highlight. We would need that LSP feature to highlight the syntax.

We could also do that inside Metals with exactly the same request for semantic highlighting. We could use fastparse and use existing parsers for json/xml etc to properly highlight things in a string. That would be available in all editors, but not on github and not without metals.

VS Code scala syntax to work we would need to embed json grammar inside the text made grammar we have.

So basically 3 options:

  • do request forwarding and allow for more features, but this will work only in VS Code
  • implement semantic highlight for multiline strings with proper interpolators, parsers already exist in Scala for a lot of what we might need. This works in all editors that support Metals.
  • work on embedding text mate grammar for json etc. This will work in VS Code and Github, maybe some editors that use the same grammar format

I think the second one is safest as we don’t have to do any tricky stuff with multiple servers or work with a scary number of regexes

26

The downside is we have to support “every” language

It’s a shame there’s not a more standard way to call “the LSP on the user’s machine associated with X”

We could also do both, we add syntax highlighting for a handful of language + we use the VS Code solution (when we’re on VS Code)

27

I’d also like to chip in that IntelliJ also has a @org.intellij.lang.annotations.Language annotation that can be used on string-typed parameters to turn on language-specific syntax for passed string literals, e.g.

  def parseTextToFile(@Language("Scala") str: String): ScalaFile
2 Likes
28

Reviving this thread to see if any progress or solution was found?

My team uses VSCode with HTML-interpolated strings (html"""<div> ... <div>""") that contain both HTML and JS code and we’d love to get syntax highlighting for these fragments.

29

I’ve not had the chance to work on it (and I might not soon)

On the other hand, it seems like the vs code-only solution should not but too much of a hassle, maybe you could try your hand at it ?

It would benefit both your team and the larger ecosystem !

30

I’ve been looking into it a bit more, and I think I had missed something important
The page I linked was about supporting full LSP features on embedded snippets
This is more powerful than the initial request of syntax highlighting, and should therefore be easier to support

And it turns out to be:

This paragraph is really short, I interpret it as meaning the task is easy, and not that it is badly documented

31

It also seems to have been done successfully for nvim:

32

Well, I did¹ it: POC for syntax highlighting for languages embedded in string interpolators by Sporarum · Pull Request #288 · scala/vscode-scala-syntax · GitHub

¹: well kinda it has a lot of limitations that are described there

3 Likes