Computerworld is undertaking a series of investigations into the world of programming languages. We've already looked at some other functional languages such as Haskell, Erlang, and (also running on the JVM) Clojure.
In this interview, we talk to Martin Odersky about Scala; a language we named as one of Six Scripting Languages your developers wish you'd let them use. Scala is one of the newer languages which runs on the Java Virtual Machine, which has become increasingly popular.
A Twitter developer has said that Scala could become the language of choice of the modern Web 2.0 startup. LinkedIn also uses the language. So do with many other big corporate names including Sony Pictures, EDF and SAP. Martin Odersky tells us about Scala’s history, its future and what makes it so interesting.
Why did you call the language Scala?
It means scalable language in the sense that you can start very small but take it a long way. For newcomers, it looks a bit like a scripting language. For the last two years we have actually been invited to compete in the JavaOne ScriptBowl, a Java scripting language competition. But Scala is not really a scripting language — that's not it's main characteristic. In fact, it can express everything that Java can and I believe there are a lot of things it can offer for large systems that go beyond the capabilities of Java. One of the design criteria was that we wanted to create a language that can be useful for everything from very small programs right up to huge systems and without the need to change structure along the way.
What led you to develop Scala?
In the 90s I became involved in the development of the Java language and its compiler. I got together with another researcher, Philip Wadler, and we developed Pizza that eventually led to Generic Java (GJ), and then to Java version 5. Along the way I got to write the javac compiler. The compiler for GJ, which was our extension, was adopted as a standard long before Sun decided to adopt the GJ language constructs into Java — they took the compiler first.
When I moved to Switzerland 10 years ago I started to work on more fundamental topics. I did some research experiments to see if we could usefully combine functional and object-oriented programming. We had tried that already in 95/96 with Pizza, but that was only a half way success because there were a lot of rough edges, which all had to do with the fact that at the time we used Java as our base language. Java was not that malleable. So starting around 2000, I developed with my group at EPFL (Ecole Polytechnique Federale de Lausanne) new languages that would continue to inter-operate with Java but that would usefully combine object-oriented and functional programming techniques.
The first of these was called Funnel and the second was called Scala. The second experiment worked out pretty well, so we decided to wrap up the experimental phase and turn Scala into a real production language that people could rely on. We polished some edges, did some minor syntax changes, rewrote the Scala tools in Scala to make sure that the language and its tools could sustain heavy usage. Then we released Scala version 2 in 2006. It's been rapidly gaining popularity since then.
What are the main benefits of combining Object-oriented and functional programming techniques?
They both bring a lot to the table. Functional programming lets you construct interesting things out of simple parts because it gives you powerful combinators -- functions that take elements of your program and combine them with other elements in interesting ways. A related benefit of functional programming is that you can treat functions as data. A typical data type in almost all programming languages is "int": you can declare an int value anywhere, including inside a function, you can pass it to a function, return it from a function or store it in a field. In a functional language, you can do the same thing with functions: declare them inside other functions, pass them into and from functions, or store them in fields. These features give you a powerful way to build your own control structures, to define truly high-level libraries, or to define new domain specific languages.
Object-oriented programming, on the other hand, offers great ways to structure your system's components and to extend or adapt complicated systems. Inheritance and aggregation give you flexible ways to construct and organise your namespaces. There's good tool support like context help in IDEs (Integrated Development Environments) that will give you pop-up menus with all the methods that you can call at a given point.
The challenge was to combine the two so that it would not feel like two languages working side by side but would be combined into one single language.
I imagine the most significant part of that challenge was in deciding what to leave out?
Yes, if you took each language style in its entirety and combined them, you would end up with a lot of duplication and you would just have two sub-languages with little interaction between them. The challenge was to identify constructs from one side with constructs from the other. For instance, a function value in a functional programming language corresponds to an object in an object-oriented language. Basically you could say that it is an object with an "apply" method. Consequently, we can model function values as objects.
Another example is in the algebraic data types of functional languages that can be modelled as class hierarchies on the object-oriented side. Also, the static fields and methods as found in Java. We eliminated these and modelled them with members of singleton objects instead. There are many other cases like these, where we tried to eliminate a language construct by matching and unifying it with something else.
What has been the overall greatest challenge you have faced in developing Scala?
Developing the compiler technology was definitely a challenge. Interestingly, the difficulties were more on the object-oriented side. It turned out that object-oriented languages with advanced static type systems were quite rare, and none of them were mainstream. Scala has a much more expressive type system than Java or similar languages, so we had to break new ground by developing some novel type concepts and programming abstractions for component composition. That led to a quite a bit of hard work and also to some new research results.
The other hard part concerned interoperability. We wanted to be very interoperable so we had to map everything from Java to Scala. There's always that tension between wanting to map faithfully the huge body of Java libraries while at the same time avoiding duplicating all constructs of the Java language. That was a persistent and challenging engineering problem. Overall I am quite happy with the result, but it has been a lot of work.
There are a lot of positive comments on forums about Scala's efficiency and scalability but another thing people often mention is that it is a very fun language to use. Was that also one of your aims in designing this language?
Absolutely. My co-workers and I spend a lot of time writing code so we wanted to have something that was a joy to program in. That was a very definite goal. We wanted to remove as many of the incantations of traditional high-protocol languages as possible and give Scala great expressiveness so that developers can model things in the ways they want to. While writing javac I did a lot of Java programming and realised how much wasted work Java programmers have to do. In Scala we typically see a two to three times reduction in the number of lines for equivalent programs. A lot of boilerplate is simply not needed. Plus it's a lot more fun to write.
This is a very powerful tool that we give to developers, but it has two sides. It gives them a lot of freedom but with that comes the responsibility to avoid misuse. Philosophically, I think that is the biggest difference between Scala and Java. Java has a fairly restrictive set of concepts so that any Java program tends to look a bit like every other Java program and it is claimed that this makes it easy to swap programmers around. For Scala, there's no such uniformity, as it is a very expressive programming language.
You can express Scala programs in several ways. You can make them look very much like Java programs which is nice for programmers who start out coming from Java. This makes it very easy for programming groups to move across to Scala, and it keeps project risks low. They can take a non-critical part first and then expand as fast as they think is right for them.
But you can also express Scala programs in a purely functional way and those programs can end up looking quite different from typical Java programs. Often they are much more concise. The benefit that gives you is that you can develop your own idioms as high-level libraries or domain specific languages embedded into Scala. Traditionally, you'd have to mix several different languages or configuration notations to achieve the same effect. So in the end, Scala's single-language approach might well lead to simpler solutions.
The learning curve for a Java developer wanting to use Scala would be quite small but how easy would it be for programmers used to working with dynamic languages with dynamic disciplines such as PHP and Python and Ruby to use?
Clearly it is easiest for a Java or .NET developer to learn Scala. For other communities, the stumbling blocks don't have so much to do with the language itself as with the way we package it and the way the tools are set up, which is Java specific. Once they learn how these things are set up, it should not be hard to learn the language itself.
What are your thoughts on Twitter using Scala? Is it good for the language's development that such a high profile site is using it?
That was great news. I am happy that they turned to Scala and that it has worked out well for them. Twitter has been able to sustain phenomenal growth, and it seems with more stability than what they had before the switch, so I think that's a good testament to Scala. When a high profile site like Twitter adopts a new language, it is really an acid test for that language. If there would be major problems with that language they'd probably be found rather quickly and highlighted prominently.
There are also a lot of other well-known companies adopting Scala. Sony Pictures Imageworks is using Scala to write its middle-tier software and Europe's largest Energy company EDF is using Scala for contract modelling in its trading arm. SAP and Siemens are using Scala in their open source Enterprise Social Messaging Experiment (ESME) tool. That's just three examples of many.
One of Twitter's developers, Alex Payne, was saying that Scala could be the language of choice for the modern web start-up and could be chosen over other languages like Python and Ruby, which have been very popular but they are not as efficient as Scala. Do you agree and did you have Web 2.0 start-ups in mind while developing Scala?
I think Scala does make sense in that space. Twitter is not the only company who has realised this; LinkedIn also uses Scala.
I think what Scala offers there is the ability to build on a solid high performance platform - the Java Virtual Machine (JVM) – while still using an agile language. There are some other options that fall into that category such as Jython, JRuby, Groovy, or Clojure, but these are all dynamically typed languages on the JVM.
In the end the question comes down to whether you are more comfortable in a statically typed setting, be it because that will catch many errors early, because it gives you a safety net for refactorings, or because it helps with performance. Or you may think you need a fully dynamic language because you want to do fancy stuff with metaprogramming. In the end it comes down to that choice. If you prefer a statically typed language, I think Scala is definitely the best option today.
What is your favourite feature of the language, if you had to pick?
I don't think I can name a single favourite feature. I'd rather pick the way Scala's features play together. For instance, how higher-order functions blend with objects and abstract types, or how actors were made possible because functions in Scala can be subclassed. The most interesting design patterns in Scala come precisely from the interaction between object-oriented and functional programming ideas.
Where do you see Scala headed in the future?
In the near term, we are currently working hard on the next release, Scala 2.8, where we are focusing on things like high performance array operations and re-defined collection libraries with fast persistent data structures, among others. That should be out by autumn [Northern Hemisphere] this year.
Then in the long term we see interesting opportunities around concurrency and parallelism, so we are looking at new ways to program multicore processors and other parallel systems. We already have a head start here because Scala has a popular actor system which gives you a high-level way to express concurrency. This is used in Twitter's message queues, for instance. The interesting thing is that actors in Scala are not a language feature, they have been done purely as a Scala library. So they are a good witness to Scala's flexibility: You can program things that will look like language features to application programmers by shipping the right kind of primitives and abstractions in a library.
We are hoping that what works for actors will also work for other concurrent abstractions such as data parallelism and stream programming. I think that in the future we will probably need several concurrency abstractions to really make use of multicore because different profiles of parallelism and concurrency will require different tools. I think that Scala's library based approach is relevant here, because it lets us mix and match concepts implemented as Scala classes and objects, thus moving forward quickly rather than having to put all of this into a language and a compiler. I think this work will keep us busy for the next four or five years.