In 1998 scientists discovered that the expansion of the universe has been accelerating

In the mid-1990s, two competing teams began observing supernovas with the goal of pinning down the rate at which the expansion of the universe was slowing down. Much to everyone’s surprise, they found just the opposite: the expansion was not slowing down, but speeding up, driven by a mysterious unseen force. In early 1998, the researchers announced these strange results that shook up the field of astrophysics.  

In 1917, as he was developing his theory of general relativity, Einstein added an arbitrary constant term to his equations in order to keep the universe static and unchanging, as it was then believed to be. Without this term, an initially static configuration of matter in the universe would tend to be pulled together under gravity; the cosmological constant was needed to counteract that tendency and keep the universe from collapsing. 

However, in 1929, Edwin Hubble looked at the redshifts of faraway galaxies and found that the rate at which an object is receding from us is proportional to that object’s distance from us. The universe was actually expanding, not static at all. The cosmological constant looked unnecessary, and Einstein then abandoned it, calling it his greatest blunder.

After Hubble’s discovery, for the next few decades most scientists believed that there was no cosmological constant. It was assumed that matter dominated the universe and would eventually cause the expansion to slow down. Depending on just how much matter there was in the universe, it might eventually collapse in a big crunch, or go on expanding forever, but more and more slowly.

Research concentrated on determining the history of the expansion of the universe by looking at extremely distant objects. Comparing the redshift of these objects with their distance gives a measure of how fast the universe is expanding.

But getting accurate distances to faraway objects is difficult. One way to do this is to find so-called standard candles, objects whose intrinsic brightness is known and thus can be compared with their apparent brightness to give a measure of their distance from us. Type Ia supernovas are just such objects. They occur when a white dwarf star that is part of a binary system attracts some extra mass from its companion star. When the white dwarf reaches a particular mass (about 1.4 times the mass of the sun), it explodes. These supernovas are extremely bright, visible billions of light years away. Since all type Ia supernovas explode when they reach the same mass, they make good standard candles. By the mid-1980s automated searches had begun to find these rare events.

In the late 1980s, a team called the Supernova Cosmology Project, led by Saul Perlmutter at Lawrence Berkeley National Laboratory, began their search for type Ia supernovas.

Starting in the mid-1990s, a second team, called the High-Z Supernova Search, led by Brian Schmidt of the Australian National University and Adam Riess of the Space Telescope Science Institute, worked on a competing effort.

The research teams used both ground-based telescopes and the Hubble Space Telescope in the race to find supernovas billions of light years away and use them to measure the (presumed) slowing of the expansion of the universe.

By late 1997, supernova data were piling up, and both groups were noticing that the distant supernovas were fainter than expected, indicating that the universe’s expansion is actually speeding up, not slowing down. 

In January 1998, at a press conference held during the Washington, DC meeting of the American Astronomical Society, the Supernova Cosmology Project team announced that they had analyzed 40 supernovas and found that the universe’s expansion would continue forever, and that the data could be explained by a cosmological constant.

After that press conference, one reporter picked up on the incredible news that there were signs of accelerating expansion and a mysterious force pushing the universe apart ever faster, while most simply reported that there would be no big crunch.

In February, the High-Z team presented their supernova data at a conference, also showing that the expansion of the universe is accelerating. Now it was clear that some strange, unseen antigravity force was driving the universe apart. Both teams soon published papers in refereed journals. These findings were completely contrary to everyone’s expectations, but with the two competing teams finding the same shocking result, they had to be taken seriously.

Later that year cosmologist Michael Turner coined the term “dark energy” to describe the mysterious force, in analogy with the invisible dark matter that makes up most of the matter in the universe.
Science magazine called the accelerating universe the “Breakthrough of the Year” in December 1998.

Now, more than ten years after the discovery, further results have confirmed that the expansion of the universe is accelerating, but the bizarre dark energy remains a mystery.

One candidate for dark energy is a cosmological constant, just as Einstein predicted (though with a different value). Quantum theory predicts that vacuum fluctuations, virtual particles that flit into and out of existence, provide energy to empty space. Unfortunately, the energy density associated with these vacuum fluctuations is, according to theoretical calculations, a whopping 120 orders of magnitude greater than the energy density cosmologists measure. Other suggestions for the dark energy have been made, and further studies are underway, but for the most part, scientists remain in the dark.

What is causing the expansion of the universe to accelerate?

What is the rate of acceleration of the expansion of the universe?

Is the universe accelerating or decelerating?

Why does dark energy make the universe accelerate?