One Wall Street firm found computational success not on the traditional path of enterprise Java, but in an obscure functional programming language called Caml, which offered the perfect tradeoff of concision and readability.
Trading firm Jane Street says Caml has given it a powerful set of tools for building large programs that have to run quickly and without errors.
“A huge amount of day-to-day programming is case analysis. Getting your programs right is really hard, and any tool you can get from the system to help catch errors is helpful,” said Yaron Minsky, head of the technology group at Jane Street, speaking Friday at the QCon developer conference in New York.
Jane Street is a proprietary trading firm that is the world’s largest industrial user of Caml and OCaml, the object-oriented version of Caml.
All of Jane Street’s trading and ancillary systems use Caml, with the exception of some C code for low-level system interfaces and some Visual Basic script powering analyst spreadsheets. All in all, Caml code handles about US$20 billion of trades every business day at Jane Street.
The company employs about 60 programmers who either knew Caml from their college days or were smart enough to be able to learn it during training. Another 90 or so non-programmers at the company use it as well.
Jane Street’s use of Caml was almost accidental, Minsky said. Caml is a dialect of the ML functional programming language, which is widely used in universities as a way to teach programming but hasn’t yet found widespread success outside academia.
Minsky started at Jane Street while still in graduate school a decade ago. Since he was familiar with Caml, he composed a number of experimental programs for the company that were later pressed into production. He got hired, too, as soon as he finished his Ph.D. in distributed systems.
At the time, Jane Street was “more trader-heavy than technology-heavy,” Minsky said. There was a lot of VB code on spreadsheets, though this approach did not scale well for the company, given that there was no system for change management to update the code across all the user’s computers.
As the company became more immersed in the world of computer trading, Minsky deployed Caml on what would become the company’s core systems.
Jane Street also looked at Java, which many trading firms use, but the way enterprise Java programs are constructed, with their dynamic object-oriented approach, can make them difficult to understand and reason about, he said.
He noted that one analyst at the company likes to review any code that runs on the system, even though he is not a programmer. He just sits down and works through the code until he understands what is going on. Caml can be easy to understand, once you get the syntax down.
Caml offers some natural advantages, too. It’s a functional language, so it provides good support for programming functions.
“You can create a function, and pass it to another function, and have that function return an array of functions,” Minsky said. Functions are “class values that can be passed around like anything else.” In many programming languages, functions cannot be passed around like values, or it takes manual work to have them passed around.
Also, data values are immutable in a functional programming language. This means programs work not by “mutating cells inside your data structures, but by the evaluation of expressions,” Minsky said.
Caml is a statically typed system, meaning there is a set of rules about each data type that the system handles. It is usually easier to catch bugs in statically typed languages than with dynamically typed systems such as JavaScript or Python. This speeds computations, because the program knows ahead of time what kind of data it is handling.
“I’ve had profoundly good experiences with programming in this style, where you use the type system to help catch the implications of your changes,” Minsky said. “You can write bad code in any language, but if you know what you are doing, you can use the type system as part of your design to greatly improve the correctness of your programs.”
While few people may have Caml on their rA(c)sumA(c)s, it has been easy to find quality programmers, Minsky said. Those who know Caml, or functional programming in general, are enthusiastic about the programming model.
The first time Minsky looked to hire programmers, he posted a notice on the Caml mailing list and ended up hiring three people from the responses.
The company also hires programmers who don’t know any Caml, reasoning — successfully, so far — that if they are smart enough they can pick up the language in training.
Over time, the company has used Caml to build not only its core trading system, but also a concurrency library, a sophisticated publish-and-subscribe system, and a collection of system automation tools.
Overall, the company has “a huge number of systems maintained by a small team,” Minsky said.
There are a number of drawbacks to using this obscure language, Minsky said. One of the major ones is lack of decent tooling. Minsky pines for the advanced editing tools, such as autocomplete, that users of popular IDEs (integrated development environments) enjoy.
To improve matters, though, the company has developed a lot of its own tools, which it has open sourced. The company has also helped to fund a company to offer commercial Caml support, called OCamlPro, and a Caml lab at Cambridge University.
“We are doing everything we can to push this language forward. Our 3 million lines of code are not going away. We depend on this language and the community around it,” Minsky said. “When you use a minority technology and are successful, you should be prepared to pay up later.”