Data

The current age is marked as the “age of data”. Due to technological advancement, datafication is everywhere and the data guys are datafying almost everything whatever we observe, we see, we talk, we photo, we search, visit or share on the web, we facebook, we instagram, we tweet, the location we visit with or without the intention of any scientific investigation. Therefore, almost anything is nothing but data. We are surrounded by and our daily life is heavily influenced by data.

In the last article Let’s meet a statistician, we have seen how data uncovered the reality in British army base hospital in Crimea. Only after exploring the data using simple statistical devices authority identified that 16,000 among 18,000 soldiers died due to causes other than the battle. Florence Nightingale appreciated the value of the data which eventually contributed to the policy changes in British hospital administration. M. C-Rundel and J. Hardin (2021) in their book “Introduction to Modern Statistics,” said,

“Scientists seek to answer questions using rigorous methods and careful observations. These observations – collected from the likes of field notes, surveys, and experiments – form the backbone of a statistical investigation and are called data.”

Some other examples of how data is influencing important decision making and thus contributing to human life and society are presented below.

Google Flu Trend (GFT) & its parable

Google receives billions of search query every second. This is a great source of data and by utilising their huge computing power Google processes all the queries continuously. They processed hundreds of billions of individual searches from 5 years of Google web search logs between 2003 and 2008 and built an automated method of discovering influenza-related search queries. The objective was to build a comprehensive model for use in influenza surveillance, with regional and state-level estimates of Influenza-like-illness (ILI) activity in the United States. They have used weekly “ILI-related physician visits” as the outcome variable and weekly count of search queries such as “medicine for cough and fever” as the only explanatory variable to generate the statistical prediction model. Based on the combination of many search queries, they built a model that could accurately estimate the current level of weekly influenza activity in each region of the United States, with a reporting lag of about one day (Ginsberg et al, 2009). Analysing search query data, Google could predict the alarm for a potential pandemic like the CDC does, however Google is even better because they could tell it in near real time. Interestingly, Google does not need to collect swabs for testing; rather, their system is built on “big data”(Kenneth & Mayer-Schönberger, 2013). It is to be noted that the model is useful for areas where a large population uses web searches. GFT is considered an exemplary use of Big Data.

Figure 1. Google flu prediction using search query data. Real percentage (black); predicted percentage (red). Source: J Ginsberg et al. Nature 000, 1-3 (2008), https://www.nature.com/articles/nature07634.

Later in 2013 GFT came to the headlines when Nature reported that Google is predicting more than double the physician visits for ILI than the proportion reported by CDC though GFT built its prediction model to predict CDC reported proportions (Lazer et al, 2014).

Figure 2 Illustration of digital health checks. Source: Lazer et al, (2014).

Lazer and colleagues identified two issues that may contribute to the GFT’s prediction mistakes: “Big Data Hubris” and “Algorithm Dynamics”. By “Big Data Hubris” they mean making assumptions about big data being a substitute rather than a supplement to the traditional data collection. Whereas by the “Algorithm Dynamics” it is meant there could be errors in measurement, changes in human behaviour, and of course continuous updating Google’s search algorithm. However, Lazer and colleagues suggested that the mistakes were avoidable by ensuring transparency, reproducibility, use of all available data and assessing the algorithm.

John Snow & Broad Street cholera outbreak

The cholera outbreak in Soho, London in August 1954 killed 616 in total within a month which is nothing compared to most recent pandemic, COVID19 though. However, this outbreak has made significant history till date. In the first 3 days 127 people living around Broad Street in Soho died and the death rate continued to grow.

Figure 3 (left) Map of London created by John Snow in 1854 showing the cholera cases. (right) John Snow memorial and public house on Broadwick Street, Soho. Source: Wikipedia/1854 Broad Street cholera outbreak

It was predominantly believed at that time that the cholera caused by polluted air “bad air” which is known as “miasma” theory. The district doctor John Snow for the first time claimed that the cholera was caused by contaminated water instead of “bad air” which was contradicting the well believed “miasma” theory. Dr Snow did a very clever thing to prove his belief. He plotted the location of the cholera cases on a map and found that the cases are clustered around the water pump. His discovery influenced public health sanitation policy in the mid-19th century. This is considered one of the earlier uses of location as data in public health problems.

Framingham Heart Study (FHS)

FHS, a phenomenal study that has generated data-based evidence laying the foundation for preventive health care since 1948s.

Md Mohiuddin

Articles by this Editor

Say, “hello Data . . .”

Articles by this Editor

Say, “hello Data . . .”