Sculpt: dependency graph extraction for Scala

Tooling
1

Sculpt is a compiler plugin for analyzing the dependency structure of Scala source code. It is now public for the first time. The source code is available under an Apache 2 license here:

The reports it generates are intended to be useful to Scala coders looking to alter the dependency structure of a codebase in order to reduce incremental compile times or split the codebase into separately buildable parts.

Sculpt uses code derived from Zinc to listen to the same dependency information coming out of the compiler that is used to guide incremental compilation.

Features include:

  • store the raw dependency data in memory as case class instances
  • output the raw dependency data as JSON
  • analyze at method level, or aggregate dependencies at class level
  • generate report of dependency layers
  • generate report of dependency cycles

See the README in the repository for more details.

There are no binary releases; the project is released as source code only.

The project is not currently actively maintained. The Scala team at Lightbend is making it available anyway in the hopes that tooling authors might find it interesting or useful.

(Thanks, Jorge, for prodding me to get this out there…)

Seth Tisue / Scala team / Lightbend, Inc.

7 Likes
2

This is really great!

Thanks for publishing this. One question: I didn’t see how to get a map of which symbols are in which files. Can this tool do that? I think for many applications we also want the mapping of file to symbol.

Thanks again!

3

ExtractDependencies.scala produces an Iterator[(Symbol, HashSet[Symbol])]. The Symbols themselves know what source files they’re from. That iterator then gets converted into instances of case class FullDependency by createFullDependencies. By design, FullDependency is outside the compiler cake and has no references to internal compiler/reflection data structures such as Symbol.

You could easily alter the FullDependency class to have fields for the source file pathnames, and alter createFullDependencies to copy that information out of the symbols.

Depending on your goals, you might also decide to simply ignore all of Sculpt except for ExtractDependencies.scala, and then just do your own thing with the Iterator[(Symbol, HashSet[Symbol])] it gives you.

Pleased to see some interest in this :slight_smile:

Seth

4

I’ll be giving a talk about Sculpt and other tools in the same space (Classycle, Acyclic, etc) at SF Scala on Feb 13. Details, abstract: https://www.meetup.com/SF-Scala/events/247363947/

2 Likes
5

slides from my talk: https://docs.google.com/presentation/d/1xgInhuw_BBJGz3UDOUktNcWPehM2UN57c9FjvzlmbRk/edit?usp=sharing

6

Johannes Rudolph sent an unfinished PR (https://github.com/lightbend/scala-sculpt/pull/51) with some improvements:

  • in class mode, treat objects and traits as classes as well so that companions are grouped together
  • simple helpers to find some sample paths Cycles.pp , useful for finding concrete cycles between two classes
  • dot generation

I can’t merge the PR as-is, but someone might like to tackle finishing up any of these three as an independent PR.

7

Chicagoans! I’ll be speaking about Sculpt and other tools in the same space (Classycle, Acyclic, etc) at Chicago Scala on Mar 29. Details, abstract: https://www.meetup.com/chicagoscala/events/248214876/

8

video of the SF Scala talk is now online https://www.youtube.com/watch?v=0nk5yphpSYQ

bonus: an 11 minute interview with me at https://www.youtube.com/watch?v=OlT9p0L76Qs talking some about Sculpt, but also more wide-ranging stuff about my Scala history, moving from Boston to San Francisco, joining Lightbend, Scala vs. Lisp, the community build, Fortify, NetLogo, and other stuff.

2 Likes