What do you think of e-ScienceCity? Click here!

Processing data

When we have large amounts of data, we need to be able to process it. Sometimes this is done just as the data is being processed, while other times the data is captured first, then processed.

The grid is a very sensible way of distributing the workload of data processing. In particle physics, the hunt for the Higgs boson relied on a network of computers across the globe processing vast amounts of data (see Finding the Higgs). You can read more about grid computing at Grid Café, if you haven’t been there yet.

The philosopher of science, Karl Popper, said that for a theory to be truly scientific, there should be a way of testing it to see whether it’s true or not, (actually, he said you have to be able to check that it isn’t true, but you can read the reason why here: http://en.wikipedia.org/wiki/Falsifiability ). However, he never said that just because we don’t have the tools yet to understand phenomena and develop an initial hypothesis, that we should just discard data we can't explain. In fact, he said that we should keep data to perform his falsification test in the future.

The value of keeping data is becoming more apparent. In 2012, new methods of analysing old data uncovered a new rocky planet (HD 40307 g) at the right distance from its parent star to have liquid water on its surface. And the mystery of the decreasing velocities of space probes Pioneer 10 and 11 was finally solved, thanks to a diligent engineer who had kept data of the crafts’ velocities for decades (see http://www.isgtw.org/feature/mystery-slowing-space-probes).

Volunteer Computing is another way large data sets can be analysed (see [Volunteer Garage]). You can donate your computer’s spare computing cycles using services such as BOINC, which connects you to a cause of your choice, whether that’s finding intelligent life on a planet like HD40307g (SETI@home) or fighting one of humanity’s biggest health challenges (FightAIDS@Home).

Another way you can volunteer to analyse data through your computer is by actively assessing images, through portals such as Galaxy Zoo, which portions out images from the huge Sloan Digital Sky Survey and allows participants to categorise galaxies based on what they look like. Galaxy Zoo has even uncovered new types of astronomical objects.

You can read more about big science and the case for the public being involved here: