Julia Speaks

Tuesday, December 11, 2018

In 2009, Viral Shah (PhD Computer Science, 2007) and Stefan Karpinski, a UC Santa Barbara computer-science graduate student, were playing catch after an Ultimate Frisbee intramural match. They may have talked some about strategy, but a more important result of their meeting was the discovery that they shared a common frustration with computer languages, what they would come to call “the two-language problem.”

“When I was at UCSB,” Shah recalls, “you typically programmed using two kinds of languages. You had languages like R, MATLAB, and Python, which are easy to use to write very high-level programs. Engineers and scientists like writing programs that are at the level of mathematical abstraction. However, those are not high-performance languages, because they cannot scale to handle big data or do large-scale simulations or big science.”

For that to be possible, a computer scientist or a programmer has to take the program written in Python or R and rewrite it in a different language, typically C, C++, or Java. The program can then run at maximum speed on a computer.

Viral Shah and Stephan Karpinski

Viral Shah and Stephan Karpinski

“To be productive and fast, you had to write the same program twice,” Shah says. “You had this mishmash of two systems. We thought it would be ideal to have a single system that could be used to write programs at a high level of abstraction — the level at which you think about the science — and have that same program run really fast on the best hardware you can lay your hands on. That is what led to the creation of Julia.”

Last August, six and a half years after Julia was first released to the scientific computing community, and with contributions from more than eight hundred programmers, the open-source language Julia 1.0 was released.

The eight hundred contributed packages, which are easy for programmers to write in Julia, dramatically expand its effectiveness by providing algorithms and methods for specific disciplines ranging from biology and chemistry to ecology, astronomy, and quantum phsyics.

In February 2019, the three eligible co-creators of the Julia language — Jeff Bezanson, Karpinski, and Shah — will receive the prestigious, quadrennially awarded James H. Wilkinson Prize for Numerical Software from the Society for Industrial Applied Mathematics (SIAM). Computer science faculty member Linda Petzold was the first UCSB recipient of the Wikinson Prize, in 1991.



To begin, only Shah (now CEO of Julia Computing) and Karpinski (CTO) knew each other, but Shah’s PhD advisor, UCSB professor John Gilbert (who was also on Karpinski's dissertation committee), had a research collaboration with Alan Edelman, a professor of mathematics at MIT and now Chief Scientist at Julia Computing. Bezanson (CTO) had just started working as a researcher in Edelman’s lab (now the Julia lab). Eventually the four developers met each other, and in 2009, they started collaborating on the new language.

“We thought we’d give it a couple of months, but when that time was up, we were having too much fun to stop,” Shah recalls. “It was a very quick and fluid interaction with like-minded people. We had a lot of ideas that look ridiculous now, and no one got upset if you broke something.”

Gilbert, whom Shah credits for “investing the time and effort in me so that I could learn everything about scientific computing that I know today,” sees in Julia’s success clear echoes of the kind of collaborative work that he has always valued as a hallmark of UCSB.

“The Julia language grew out of an interdisciplinary mix of computational science, performance computing, and numerical linear algebra, together with twenty-first-century advances in programming languages and compilers,” says Gilbert. “Because of how it combines ease of use with high performance, Julia has taken an important place in data science and machine learning. It's great to see my students' success; we can all be proud of UCSB's unique ethos and history of nurturing this sort of cross-disciplinary research.”

Shah says that starting with four different disciplines helped the team avoid tunnel vision: “Alan is a mathematician at MIT, Stefan is a data scientist, Jeff is a programming languages and computer-science researcher, and I am a computational scientist. All of us believe that computing can solve the world’s toughest problems, and that creating a high-performance programming language that builds bridges between diverse academic communities was the gateway to solving grand challenges. As a result, there was a natural mixing of diverse ideas and viewpoints, even as we all agreed on the big picture.”

The team leveraged multiple open-source projects that predated Julia, resurrected a number of ideas that dated to the 1960s but had fallen out of favor during the PC revolution, and used much of that material in new ways while combining it with their own novel ideas.

The core principles were established by 2012, when Karpinski wrote a blog post that read in part, “If you are a greedy, demanding, unreasonable programmer, we invite you to give it a try.”

At that time, they expected to release version 1.0 in a few months, but that turned into six years. “That's because in an open-source project you can't just introduce a new version some years down the line and slap on a new version number,” Shah explains. “We started this to solve the real problem and wanted to be sure that we had a solid foundation for everyone to go out and start using. That's why it took six years.”

The proof of the team’s success lies in the number of high-profile, high-impact applications in which Julia is being used.

The Federal Aviation Administration is using Julia to develop the next generation of its Airborne Collision Avoidance System. When two aircraft are within a minute of a midair collision, the system gives the pilots of both aircraft warnings and tells them what corrective action to take to avoid colliding. The current system is quite old, and a team at Lincoln Labs at Johns Hopkins University is using Julia to design its new replacement. The project requires computation of an exhaustive search comprising 650 billion decision points. Julia reduced the time required to conduct those computations by several years.

In 2015 economists at the Federal Reserve Bank of New York used Julia to publish the bank’s most comprehensive, most complex macroeconomic models, known as Dynamic Stochastic General Equilibrium. Invenia Technical Computing employs the latest research in machine learning, complex systems, risk analysis, and energy systems to optimize the electrical grid across North America. Its current codebase is written mostly in MATLAB, Python, and C. Now Invenia is looking to scale up its operations, and their language of choice for the experiment is Julia.

Julia is used in energy analytics and optimization, medical modeling of cancer evolution, mapping genetic diversity, optimizing milk output, risk assessment using large-scale Monte Carlo simulations, robotics, space-mission planning, schoolbus route optimization, and much more.

Because computers are not getting much faster today, large computing tasks may be handled by harnessing the power of millions of “commodity” computers — i.e. standard laptops — in a “distributed system” the total power of which is similar to that of a supercomputer.

In one such Julia project, called Celeste, Julia was used to write the program to catalog images of some 188 million astronomical objects that have been captured since 1998 by telescopes at Arizona’s Apache Point Observatory, as part of the Digital Sky Survey. The images contained 60 terabytes of data. The data was loaded onto the supercomputer Cori at the Lawrence Berkeley National Laboratory, then the world's fifth-largest computer, consisting of 650,000 compu­ter cores.

The data analysis program, written in Julia, which makes distributed computing easy, was able to catalogue the 188 million light sources in less than 15 minutes,” Viral says. “On a single laptop, it would have taken I don't know how many years.”

“When the Julia project got started in 2009, the project’s goal of unifying high performance and high productivity seemed like a far-off dream,” says Keno Fischer, a chief technology officer at Julia Computing, on the Julia website. “In 2017, projects like Celeste show that this dream has become a reality.

"Scientists can now take the prototypes they have developed on their laptops and run them on the biggest supercomputers without having to switch languages or completely rewrite their code. We are confident that Julia will help advance the frontier of scientific discovery for many years to come.”

Shah concludes, “When we started we were just doing something that the four of us wanted for our own purposes. We were fed up with the way things were being done. We never imagined it would come this far.”

Julia Speaks