dotty indent syntax - I found it easy once I used it

I have seen alot of github issue comments and postings around dotty’s significant whitespace syntax proposed change. I wanted to write this note to record my experince with it based on hard, grinding daily use. “Hard, grinding” means I needed to quickly spin up a PoC with multiple “bigdata” data ingest pipelines and scala was part of that solution along with me hacking on python, NLP (bert’ish things), elasticsearch, neo4j and a few other technologies (UIs are painful).

For me, the new syntax has been easy to adopt. Here’s what I did to start using it:

• Read the dotty web site page (which links to the github page describing the syntax). It took 15 minutes to read it as the rules are simple and short.
• Ran the rewrite command using sbt. I added the rewrite command flags, ran compile, then changed the flags for the 2nd rewrite. I had to ask the gitter channel on how to do this with sbt as it requires two manual runs of sbt changing the flags between runs.
• Corrected one rewrite error out of 4K lines of code. The rewriter missed a trailing comma in an enum definition.
• Added “-indent” to the compiler flags going forward.

All of these steps took about an hour. I use way too much python, am not a PL purist, and mostly care about what works, and makes it easy for me to get work done.

Adapting to the new syntax took one hour. I had to try a few ways of using it for some flatMaps and maps to get the style I wanted, and I noticed that when keeping multiple statements on one line, I still used braces. Overall, dotty conversion is clocking in at 4 hours total (see my previous post for the other two hours), mostly on my own and on an experimental compiler. To put that into context, it took me a year to learn the hardcore FP programming style. It would have been nice to have someone who had already gone through this to explain it to me or at least more blogs on the topic so I can use proven steps.

I quickly become highly productive with the new syntax. That’s two hours or the syntax switch so what do I think about it? Was it worth it to me?

Here’s what I think based on using it intensely:

• I did not have any mental burden switching to it and thought that it helped improve the appearance of my code.
• I did not have any cut and paste issues. I cut and pasted a lot of code around because I tend to write larger functions to get the logic down then break the function down into pieces to make it testable and reusable (e.g., more generic). Just like with python, I re-indent the code. Editors make this easy and its just like cutting and pasting python code.
• I liked the “end <thing>” syntax. It was much better than many braces.
• It was straightforward except around multiple parameter lists as noted below.
• I did not have any cognitive dissonance when reading code written with the old syntax. Of course, I know the old brace syntax.
• Git diffs are still easy to read. I like seeing the entire file at one time so I tend to read diffs in the context of github. I find large git diffs at the command line hard to understand regardless of syntax whether indentation, or braces.
• I think three spaces are about right, although four could work for me. It makes the indentation easy to read.
• It vertically compressed my code so I could see more code on the screen at once without annoying braces tree’d out across the screen.

I overwhelmingly prefer the new syntax. I did not realize how much scala code was littered with less-than-helpful braces. It was worth the switch in my simple case.

I do not think the new syntax is “named” well as I felt the name implies you can use it everywhere like with python. It cannot be used it on folds with two parameter lists. It is not perfect. I would really like to use it everywhere. I did not have any problems adapting to it–really! I just need to know what works where and then I adapt to it–I have to focus as much as possible on the domain problem. You cannot use it inside (inline) function arguments because you cannot place the comma correctly. More baking time is needed and I would like to see these areas addressed. Based on my hands-on experiences, I do not consider these issues to be showstoppers.

The syntax appears to be controversial. Many issues relate to change management, but other issues relate to technical usage concerns. I was not able to substantiate any of the technical concerns, but I know that change management is important, so more experience is necessary.

People also seem concerned that this change is distracting the scala compiler team form more critical matters such as getting the new compiler completed and polished. While it is easy to agree with that thought, I think that changes to increase adoption by new programmers and other non-technical concerns are important. I do not have the exposure to judge whether this change will help broaden scala’s appeal, but if there is a chance that it does, it seems worth an experiment or two. I hope there are other dotty features that help expand the scala universe beyond backend processing. Expansion is important, appears not to have been a top priority in the past, but appears to be changing re: scala-js. I would like to see an expanded set of scala users so that the eco-system grows. A rising tide lifts all boats.

Let’s assume change management issues exist around changing to the indentation syntax in the first release of dotty in a year. Are they significant enough to outweigh the benefits? I think that’s a hard question to answer a-priori. We need data based on hands-on experience. Given dotty’s schedule, it looks like there is time to explore the issues and respond accordingly.