Lisp, javascript, other programming languages...a thought
While I'm not a lisp (common lisp, scheme'ish) programming guru, I occasionally use lisp/scheme to write programs for analysis or data management tasks. Since I am data mining/data scientist person, I write many smaller programs to build out analysis workflows. I use many different tools although I would prefer just one, or at the very least, fewer.
Sometimes I need to write some web code, and for those tasks, javscript is the dominant language. Javascript has a history similar to lisp but a future that is probably different given the attention and focus it has. I was recently updating my skills in some of the "web" programming toolkits, e.g. moving from grunt to webpack, and I had a thought. I'll admit that I had this thought even though I once again felt that the javascript ecosystem has too many pieces to it--a great source of innovation *and* irritation.
Javascript was stuck with the same, rather poor, language standard for a decade until recent committee experts with significant financial backing were able to get the consensus process moving again. The language was updated many years ago but it took a long time to gain consensus get frontend browser and backend server systems to update their software to be more adherent.
In the gap years, several projects sprung up to produce enhancements to the javascript language. Some enhancements, like typescript, adds more type safety to javascript and is a thin layer over javascript. Others, such as scalajs, add more thickness since they are moving an entire language. Other languages such as haskell, clojure, elm, scheme and common lisp have been rebuilt or specifically targeted to run on top of javascript.
The "new" languages often use the term "transpiler" to describe the process that takes the higher level language and converts it into javascript. The more common term compiler is sometimes used this way although it usually desribes the process of translating source code to machine code.
Interestingly enough, a "webassembly" effort just standardized a new "virtual machine"-like specification to allow more languages to be created on top of javascript easier as well as, and more importantly, to make javascript chunks of code more efficient to load and execute.
It is testamount to javscript's power that a "babel" transpiler can mostly express new javascript versions in older javascript standards. Although many of the updates to javascript are relatively minor syntax updates, some are more interesting. It is clear that the javascript ecosystem is evolving very rapidly.
But the rapid evolution is a bit of deja-vu since what is new in the javascript world happened a long time ago with java, lisp and eventually .Net--all of these languages/environments have had virtual machines for a very long time. Lisp especially, relies on many small packages to plug holes and cover some bumps in the javascript language just like the multitude of javascript packages perform the same role.
The javascript advances are good for all of us. However, it does makes me think about lisp and why lisp, although stuck from a specification evolution perspective (with scheme being only slightly better), solves many of the "workarounds" that are present in the giant javascript ecosystem.
For example, lispers believe that macros are an essential part to building a Domain Specific Language (DSL) that reduces the friction from the problem domain to code. I think this is true. If we look at the javascript community, we see that transpilers and the like are really taking the place of macros in lisp--transpilers manipulate source code before evaluation. Granted, the transpilers in javascript operate at the language level vs the domain abstraction level, but the point I make below is still valid even if you include many of the javascript frameworks that are more domain specific.
Lisp macros are like transpilers. To me, lisp macros can be focused on the small (function level) or large (DSL creation) and allow you slice lisp into a DSL with a targeted granularity. Transpilers really force you to be able parse the entire javascript language--not a huge deal, but more work than just creating a small macro built into the language. A language with macros allows you to evolve the language as a community or an individual. A transpiler, given its focus on the entire language, takes a community.
Hence, at the level of language granularity, transpilers are really much like large macros and we can witness the power of "macros" by seeing how important they are with javascript. Scala, for example, has macros that can operate on the scala AST and target different levels of grain and its a useful tool. Javascript has transplilers for large language level issues and some smaller e.g. JSX. I admit that I am glossing over some of the nice things about lisp macros that make lisp macros even more interesting e.g. compile time and runtime type checking.
The use of transpilers has a rich history. Massively Parallel Processing (MPP) vendors (KSR, TMC) had the high level language back end (HLLBE) for parallel processing and there are many other examples where code transformers (i.e. macros or transpilers) made a difference. Their usefulness is really unquestioned by most people today although I think most people would agree that it would be nice to not have to use them so explicitly in their projects.
Macros/transpilers allow you to different things and in fact, these types of tools allow you to control evaluation (lazy/strict), binding, syntax changes and automatic code generation. We see that some of these capabilities are more easily handled in macros/transpilers and some of these capabilities can be implemented with specific language features if available. Python, for example, gets along Ok without macros but monkey patching and other features of the language to make up for it. As you would suspect, Python's features only allow you to take the "outputs" of macros so far. Python needed an update to the transaltor/compiler to allow some syntax changes e.g. destructuring. Javascript used a transpiler to allow destructuring initially in babel. Lisp already had destructuring and the macro system can be used to extend it, as needed, for some DSLs.
There is a balance between the ecosystem of tools, culture, community (community means many things here), costs and the features a programming language provides. These dimensions can be evaluated for the "job" at hand. Here a "job" is a combination of what needs to get done, the benefit of getting the job done as well "how" the job needs to get done.
The ecosystem and programming language can be difficult to analyze. In the javascript world, the transpiler performs the same job as a lisp macro for some things. Its important to recognize that a language may lack a feature, say macros, but an equivalent or good-enough feature may exist in the ecosystem, say a transpiler. A refined analysis may even suggest that the macro/transpiler capability is equivalent and easy-enough-to-use regardless of where that capability is found. Another person's analysis, taking into account the skill level of the team, may suggest that a macro-system-in-the-language is easier. You do have to be fair in the analysis though. A feature in one language, could be covered by an ecosystem capability in another set.
Since the reality is that we do not in general care about programming languages per se and we care more about getting the "job" done (whether commercial or academic) we must evaluate the intersection of the ecosystem and specific language features in order to select the best set of tools to solve a problem. If we take this "jobs" lens, then indeed, even though javascript has some foolishness in the language itself, the ecosystem in its current form, is more than making up for it and javascript is fairly powerful in its ability to be adapted to different styles that programmers may prefer.
It's quite possible that a programming language developed in an academic setting can make the jump to commercial setting if, for example, it initially lacks a build system but a build system is developed in the commercial world that nullifies solves the build system problem. It is important to recognize that other types of ecosystem issues may forever prevent a programming language or ecosystem from jumping between worlds easily. Its possible that a programming language is designed to address a very specific problem in academia and that it is too narrow, regardless of ecosystem changes, to be applied to the commercial world.
For example, if a specific "job" requires significant inter-process/computing node communication and extreme reliability, a language *and* ecosystem that supports that model of solution will be useful i.e. erlang for distributed computing with high reliability in the telecommunications equipment industry. In this case, a language that composes at the function level really well, is less important than an ecosystem+language that composes across process boundaries well and some people have determined that erlang is a good answer.
If the job is being a programming language for browsers, where browsers are not standardized, run on different form factors and have different applications running in them, then a language *and* ecosystem that enables extreme adaption makes alot of sense.
If the job is to program a front- and back-end and I want to minimize the impact of having multiple programming languages, I might choose a language that can be hosted in a browser and the back-end *and* has sufficient tooling and libraries in both to get the job done under a specific target cost.
It is easy to leave out a dimension in the language or ecosystem when evaluating the trade-offs for a job. Most arguments I see on the internet about this-or-that feature of a programming language are because the participants have not really agreed to the "job" or an evaluation framework. The arguments should really ensure that the the two sides of the coin (ecosystem/language and the job) are agreed to first. The argument about how to evaluate the dimensions, the "score" given to a dimension and how to add the scores together are worth significant discussion in my opinion.
I find that many arguments that involve commercial "jobs" leave out the total cost over time of a specific choice of ecosystem and programming language. I do admit that estimating the long term costs for some choices is very hard to do well.
So the key thought is really quite obvious. Looking at specific programming languages is not helpful, as its the intersection of the ecosystem and language that help you get jobs done. Trade-offs are made across the key dimensions of the language and ecosystem. These trade-offs should be evaluated for the target "job" to help you make an optimal set of trade-offs and in the end, a decision.
Sometimes I need to write some web code, and for those tasks, javscript is the dominant language. Javascript has a history similar to lisp but a future that is probably different given the attention and focus it has. I was recently updating my skills in some of the "web" programming toolkits, e.g. moving from grunt to webpack, and I had a thought. I'll admit that I had this thought even though I once again felt that the javascript ecosystem has too many pieces to it--a great source of innovation *and* irritation.
Javascript was stuck with the same, rather poor, language standard for a decade until recent committee experts with significant financial backing were able to get the consensus process moving again. The language was updated many years ago but it took a long time to gain consensus get frontend browser and backend server systems to update their software to be more adherent.
In the gap years, several projects sprung up to produce enhancements to the javascript language. Some enhancements, like typescript, adds more type safety to javascript and is a thin layer over javascript. Others, such as scalajs, add more thickness since they are moving an entire language. Other languages such as haskell, clojure, elm, scheme and common lisp have been rebuilt or specifically targeted to run on top of javascript.
The "new" languages often use the term "transpiler" to describe the process that takes the higher level language and converts it into javascript. The more common term compiler is sometimes used this way although it usually desribes the process of translating source code to machine code.
Interestingly enough, a "webassembly" effort just standardized a new "virtual machine"-like specification to allow more languages to be created on top of javascript easier as well as, and more importantly, to make javascript chunks of code more efficient to load and execute.
It is testamount to javscript's power that a "babel" transpiler can mostly express new javascript versions in older javascript standards. Although many of the updates to javascript are relatively minor syntax updates, some are more interesting. It is clear that the javascript ecosystem is evolving very rapidly.
But the rapid evolution is a bit of deja-vu since what is new in the javascript world happened a long time ago with java, lisp and eventually .Net--all of these languages/environments have had virtual machines for a very long time. Lisp especially, relies on many small packages to plug holes and cover some bumps in the javascript language just like the multitude of javascript packages perform the same role.
The javascript advances are good for all of us. However, it does makes me think about lisp and why lisp, although stuck from a specification evolution perspective (with scheme being only slightly better), solves many of the "workarounds" that are present in the giant javascript ecosystem.
For example, lispers believe that macros are an essential part to building a Domain Specific Language (DSL) that reduces the friction from the problem domain to code. I think this is true. If we look at the javascript community, we see that transpilers and the like are really taking the place of macros in lisp--transpilers manipulate source code before evaluation. Granted, the transpilers in javascript operate at the language level vs the domain abstraction level, but the point I make below is still valid even if you include many of the javascript frameworks that are more domain specific.
Lisp macros are like transpilers. To me, lisp macros can be focused on the small (function level) or large (DSL creation) and allow you slice lisp into a DSL with a targeted granularity. Transpilers really force you to be able parse the entire javascript language--not a huge deal, but more work than just creating a small macro built into the language. A language with macros allows you to evolve the language as a community or an individual. A transpiler, given its focus on the entire language, takes a community.
Hence, at the level of language granularity, transpilers are really much like large macros and we can witness the power of "macros" by seeing how important they are with javascript. Scala, for example, has macros that can operate on the scala AST and target different levels of grain and its a useful tool. Javascript has transplilers for large language level issues and some smaller e.g. JSX. I admit that I am glossing over some of the nice things about lisp macros that make lisp macros even more interesting e.g. compile time and runtime type checking.
The use of transpilers has a rich history. Massively Parallel Processing (MPP) vendors (KSR, TMC) had the high level language back end (HLLBE) for parallel processing and there are many other examples where code transformers (i.e. macros or transpilers) made a difference. Their usefulness is really unquestioned by most people today although I think most people would agree that it would be nice to not have to use them so explicitly in their projects.
Macros/transpilers allow you to different things and in fact, these types of tools allow you to control evaluation (lazy/strict), binding, syntax changes and automatic code generation. We see that some of these capabilities are more easily handled in macros/transpilers and some of these capabilities can be implemented with specific language features if available. Python, for example, gets along Ok without macros but monkey patching and other features of the language to make up for it. As you would suspect, Python's features only allow you to take the "outputs" of macros so far. Python needed an update to the transaltor/compiler to allow some syntax changes e.g. destructuring. Javascript used a transpiler to allow destructuring initially in babel. Lisp already had destructuring and the macro system can be used to extend it, as needed, for some DSLs.
There is a balance between the ecosystem of tools, culture, community (community means many things here), costs and the features a programming language provides. These dimensions can be evaluated for the "job" at hand. Here a "job" is a combination of what needs to get done, the benefit of getting the job done as well "how" the job needs to get done.
The ecosystem and programming language can be difficult to analyze. In the javascript world, the transpiler performs the same job as a lisp macro for some things. Its important to recognize that a language may lack a feature, say macros, but an equivalent or good-enough feature may exist in the ecosystem, say a transpiler. A refined analysis may even suggest that the macro/transpiler capability is equivalent and easy-enough-to-use regardless of where that capability is found. Another person's analysis, taking into account the skill level of the team, may suggest that a macro-system-in-the-language is easier. You do have to be fair in the analysis though. A feature in one language, could be covered by an ecosystem capability in another set.
Since the reality is that we do not in general care about programming languages per se and we care more about getting the "job" done (whether commercial or academic) we must evaluate the intersection of the ecosystem and specific language features in order to select the best set of tools to solve a problem. If we take this "jobs" lens, then indeed, even though javascript has some foolishness in the language itself, the ecosystem in its current form, is more than making up for it and javascript is fairly powerful in its ability to be adapted to different styles that programmers may prefer.
It's quite possible that a programming language developed in an academic setting can make the jump to commercial setting if, for example, it initially lacks a build system but a build system is developed in the commercial world that nullifies solves the build system problem. It is important to recognize that other types of ecosystem issues may forever prevent a programming language or ecosystem from jumping between worlds easily. Its possible that a programming language is designed to address a very specific problem in academia and that it is too narrow, regardless of ecosystem changes, to be applied to the commercial world.
For example, if a specific "job" requires significant inter-process/computing node communication and extreme reliability, a language *and* ecosystem that supports that model of solution will be useful i.e. erlang for distributed computing with high reliability in the telecommunications equipment industry. In this case, a language that composes at the function level really well, is less important than an ecosystem+language that composes across process boundaries well and some people have determined that erlang is a good answer.
If the job is being a programming language for browsers, where browsers are not standardized, run on different form factors and have different applications running in them, then a language *and* ecosystem that enables extreme adaption makes alot of sense.
If the job is to program a front- and back-end and I want to minimize the impact of having multiple programming languages, I might choose a language that can be hosted in a browser and the back-end *and* has sufficient tooling and libraries in both to get the job done under a specific target cost.
It is easy to leave out a dimension in the language or ecosystem when evaluating the trade-offs for a job. Most arguments I see on the internet about this-or-that feature of a programming language are because the participants have not really agreed to the "job" or an evaluation framework. The arguments should really ensure that the the two sides of the coin (ecosystem/language and the job) are agreed to first. The argument about how to evaluate the dimensions, the "score" given to a dimension and how to add the scores together are worth significant discussion in my opinion.
I find that many arguments that involve commercial "jobs" leave out the total cost over time of a specific choice of ecosystem and programming language. I do admit that estimating the long term costs for some choices is very hard to do well.
So the key thought is really quite obvious. Looking at specific programming languages is not helpful, as its the intersection of the ecosystem and language that help you get jobs done. Trade-offs are made across the key dimensions of the language and ecosystem. These trade-offs should be evaluated for the target "job" to help you make an optimal set of trade-offs and in the end, a decision.
Comments
Post a Comment