View source for Google Flu Trends

{{short description|Former web service operated by Google}}
[[File:Google Flu Trends Data, South Africa.png|thumb|upright=1.3|Google Flu Trends data, South Africa]]
'''Google Flu Trends''' ('''GFT''') was a [[web service]] operated by [[Google]]. It provided estimates of [[influenza]] activity for more than 25 countries. By aggregating [[Google Search]] queries, it attempted to make accurate predictions about flu activity. This project was first launched in 2008 by Google.org to help predict outbreaks of flu.<ref name=google-faq />

Google Flu Trends stopped publishing current estimates on 9 August 2015. Historical estimates are still available for download, and current data are offered for declared research purposes.<ref name=pcworld>{{cite web|url=http://www.pcworld.com/article/2974153/websites/google-flu-trends-calls-out-sick-indefinitely.html|title=Google Flu Trends calls out sick, indefinitely|author=Fred O'Connor|date=20 August 2015|work=PCWorld|access-date=23 August 2015|archive-url=https://web.archive.org/web/20150823220808/http://www.pcworld.com/article/2974153/websites/google-flu-trends-calls-out-sick-indefinitely.html|archive-date=23 August 2015|url-status=live}}</ref>

==History==
The idea behind Google Flu Trends was that, by monitoring millions of users’ health tracking behaviors online, the large number of Google search queries gathered can be analyzed to reveal if there is the presence of flu-like illness in a population. Google Flu Trends compared these findings to a historic baseline level of influenza activity for its corresponding region and then reports the activity level as either minimal, low, moderate, high, or intense. These estimates have been generally consistent with conventional surveillance data collected by health agencies, both nationally and regionally.

[[Roni Zeiger]] helped develop Google Flu Trends.<ref>{{cite web | url=https://www.youtube.com/watch?v=6111nS66Dpk | title=Google Flu Trends Overview | publisher=[[YouTube]] | first=Roni | last=Zeiger | website=youtube.com | date=6 October 2009 | accessdate=6 June 2013 | archive-url=https://web.archive.org/web/20130606164710/http://www.youtube.com/watch?v=6111nS66Dpk | archive-date=6 June 2013 | url-status=live }}</ref>

==Methods==
Google Flu Trends was described as using the following method to gather information about flu trends.<ref name=":0">{{cite journal|last=Ginsberg|first=Jeremy|title=Detecting influenza epidemics using search engine query data|journal=Nature |year=2009 |volume=457 |issue=7232 |pages=1012–1014 |doi=10.1038/nature07634 |pmid=19020500 |bibcode=2009Natur.457.1012G |s2cid=125775 |url=http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/archive/papers/detecting-influenza-epidemics.pdf|accessdate=10 November 2012|archive-url=https://web.archive.org/web/20120904220105/http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/archive/papers/detecting-influenza-epidemics.pdf|archive-date=4 September 2012|url-status=live}}</ref><ref>{{cite journal|last1=Ginsberg|first1=Jeremy|last2=Mohebbi|first2=Matthew H.|last3=Patel|first3=Rajan S.|last4=Brammer|first4=Lynnette|last5=Smolinski|first5=Mark S.|last6=Brilliant|first6=Larry|title=Detecting influenza epidemics using search engine query data|journal=Nature|volume=457|pages=1012–1014|date=19 February 2009|doi=10.1038/nature07634|pmid=19020500|issue=7232|bibcode=2009Natur.457.1012G|s2cid=125775|doi-access=free}}</ref>

First, a time series is computed for about 50 million common queries entered weekly within the United States from 2003 to 2008. A query's time series is computed separately for each state and normalized into a fraction by dividing the number of each query by the number of all queries in that state. By identifying the IP address associated with each search, the state in which this query was entered can be determined.

A linear model is used to compute the log-odds of Influenza-like illness (ILI) physician visit and the log-odds of ILI-related search query: 
:<math>\operatorname{logit}(P) = \beta_0 + \beta_1 \times \operatorname{logit}(Q) + \epsilon</math>
''P'' is the percentage of ILI physician visit and ''Q'' is the ILI-related query fraction computed in previous steps. β<sub>0</sub> is the intercept and β<sub>1</sub> is the coefficient, while ε is the error term.<ref>{{Cite web |date=12 August 2014 |title=Using Clinicians’ Search Query Data to Monitor Influenza Epidemics |url=https://pmc.ncbi.nlm.nih.gov/articles/PMC4296132/ |url-status=live |archive-date= |access-date= |website=pmc |doi=10.1093/cid/ciu647}}</ref>

Each of the 50 million queries is tested as ''Q'' to see if the result computed from a single query could match the actual history ILI data obtained from the U.S. Centers for Disease Control and Prevention (CDC). This process produces a list of top queries which gives the most accurate predictions of CDC ILI data when using the linear model. Then the top 45 queries are chosen because, when aggregated together, these queries fit the history data the most accurately. Using the sum of top 45 ILI-related queries, the linear model is fitted to the weekly ILI data between 2003 and 2007 so that the coefficient can be gained. Finally, the trained model is used to predict flu outbreak across all regions in the United States.

This algorithm has been subsequently revised by Google, partially in response to concerns about accuracy, and attempts to replicate its results have suggested that the algorithm developers "felt an unarticulated need to cloak the actual search terms identified".<ref name=Science>{{cite journal |last1= Lazer |first1= David |last2= Kennedy |first2= Ryan |last3= King |first3= Gary |last4= Vespignani |first4= Alessandro |date= 14 March 2014 |title= The Parable of Google Flu: Traps in Big Data Analysis |journal= Science |volume= 343 |issue= 6176 |pages= 1203–1205 |doi= 10.1126/science.1248506 |pmid= 24626916 |bibcode= 2014Sci...343.1203L |s2cid= 206553739 |url= https://dash.harvard.edu/bitstream/handle/1/12016836/The%20Parable%20of%20Google%20Flu%20%28WP-Final%29.pdf?sequence=1 |access-date= 19 December 2019 |archive-url= https://web.archive.org/web/20191119022520/https://dash.harvard.edu/bitstream/handle/1/12016836/The%20Parable%20of%20Google%20Flu%20%28WP-Final%29.pdf?sequence=1 |archive-date= 19 November 2019 |url-status= live }}</ref>

== Privacy concerns ==
Google Flu Trends tries to avoid privacy violations by only aggregating millions of anonymous search queries, without identifying individuals that performed the search.<ref name=google-faq>{{cite web|title=Google Flu Trends {{!}} How|url=http://www.google.org/flutrends/about/how.html|accessdate=10 November 2012|archive-url=https://web.archive.org/web/20121022154915/http://www.google.org/flutrends/about/how.html|archive-date=22 October 2012|url-status=live}}</ref><ref name=NYT-privacy>{{cite news|title=Is There a Privacy Risk in Google Flu Trends?|url=http://bits.blogs.nytimes.com/2008/11/13/does-google-flu-trends-raises-new-privacy-risks/|accessdate=10 November 2012|work=The New York Times|first=Miguel|last=Helft|date=13 November 2008|archive-url=https://web.archive.org/web/20121106055230/http://bits.blogs.nytimes.com/2008/11/13/does-google-flu-trends-raises-new-privacy-risks/|archive-date=6 November 2012|url-status=live}}</ref> Their search log contains the IP address of the user, which could be used to trace back to the region where the search query is originally submitted. Google runs programs on computers to access and calculate the data, so no human is involved in the process. Google also implemented the policy to anonymize IP address in their search logs after 9 months.<ref>{{cite web|title=Privacy Policy – Policies & Principles – Google|url=http://www.google.com/intl/en-US/policies/privacy/|accessdate=10 November 2012|archive-url=https://web.archive.org/web/20121103041504/http://www.google.com/intl/en-US/policies/privacy/|archive-date=3 November 2012|url-status=live}}</ref>

However, Google Flu Trends has raised privacy concerns among some privacy groups. [[Electronic Privacy Information Center]] and Patient Privacy Rights sent a letter to [[Eric Schmidt]] in 2008, then the CEO of Google.<ref>{{cite web|last=Peel|first=Deborah|title=EPIC's November 12, 2008 Letter to Google Concerning Google Flu Trends|url=http://epic.org/privacy/flutrends/EPIC_ltr_FluTrends_11-08.pdf|accessdate=10 November 2012|archive-url=https://web.archive.org/web/20121115055603/http://epic.org/privacy/flutrends/EPIC_ltr_FluTrends_11-08.pdf|archive-date=15 November 2012|url-status=live}}</ref> They conceded that the use of user-generated data could support public health effort in significant ways, but expressed their worries that "user-specific investigations could be compelled, even over Google's objection, by court order or Presidential authority".

==Impact==
An initial motivation for GFT was that being able to identify disease activity early and respond quickly could reduce the impact of seasonal and pandemic influenza.  One report was that Google Flu Trends was able to predict regional outbreaks of flu up to 10 days before they were reported by the CDC (Centers for Disease Control and Prevention).<ref name=NYT>{{cite web|title=Google Uses Searches to Track Flu's Spread|url=http://msl1.mit.edu/furdlog/docs/nytimes/2008-11-11_nytimes_google_influenza.pdf|accessdate=10 November 2012|archive-url=https://web.archive.org/web/20121115062114/http://msl1.mit.edu/furdlog/docs/nytimes/2008-11-11_nytimes_google_influenza.pdf|archive-date=15 November 2012|url-status=live}}</ref>

In the [[2009 flu pandemic]] Google Flu Trends tracked information about flu in the United States.<ref>{{Cite journal | last1 = Cook | first1 = S. | last2 = Conrad | first2 = C. | last3 = Fowlkes | first3 = A. L. | last4 = Mohebbi | first4 = M. H. | editor1-last = Cowling | editor1-first = Benjamin J | title = Assessing Google Flu Trends Performance in the United States during the 2009 Influenza Virus A (H1N1) Pandemic | doi = 10.1371/journal.pone.0023610 | journal = PLOS ONE | volume = 6 | issue = 8 | pages = e23610 | year = 2011 | pmid =  21886802| pmc =3158788 | bibcode = 2011PLoSO...623610C | doi-access = free }}</ref> In February 2010, the CDC identified influenza cases spiking in the mid-Atlantic region of the United States. However, Google's data of search queries about flu symptoms was able to show that same spike two weeks prior to the CDC report being released.{{citation needed|date=August 2022}}

“The earlier the warning, the earlier prevention and control measures can be put in place, and this could prevent cases of influenza,” said Dr. [[Lyn Finelli]], lead for surveillance at the influenza division of the CDC. “From 5 to 20 percent of the nation's population contract the flu each year, leading to roughly 36,000 deaths on average.” <ref name=NYT />

Google Flu Trends is an example of [[collective intelligence]] that can be used to identify trends and calculate predictions. The data amassed by search engines is significantly insightful because the search queries represent people's unfiltered wants and needs. “This seems like a really clever way of using data that is created unintentionally by the users of Google to see patterns in the world that would otherwise be invisible,” said Thomas W. Malone, a professor at the Sloan School of Management at MIT. “I think we are just scratching the surface of what's possible with collective intelligence.” <ref name=NYT />

==Accuracy==
The initial Google paper stated that the Google Flu Trends predictions were 97% accurate comparing with CDC data.<ref name=":0" /> However subsequent reports asserted that Google Flu Trends' predictions have been very inaccurate, especially in two high-profile cases. Google Flu Trends failed to predict the 2009 spring pandemic<ref>{{Cite journal |last=Schmidt |first=Charles |date=2019-09-18 |title=Real-time flu tracking |url=https://www.nature.com/articles/d41586-019-02755-6 |journal=Nature |language=en |volume=573 |issue=7774 |pages=S58–S59 |doi=10.1038/d41586-019-02755-6}}</ref> and over the interval 2011–2013 it consistently overestimated relative flu incidence,<ref name=Science/> predicting twice as many doctors' visits over one interval in the 2012-2012 flu season as the CDC recorded.<ref name="Science"/><ref>{{cite journal |last= Butler |first=Declan|date= 13 February 2013|title= When Google got flu wrong |journal=Nature |volume=494 |issue=7436|pages= 155–156 |doi= 10.1038/494155a |pmid=23407515|bibcode=2013Natur.494..155B|doi-access= free }}</ref> A 2022 study published (with commentaries) in the ''International Journal of Forecasting''<ref>{{Cite journal |last1=Katsikopoulos |first1=Konstantinos V. |last2=Şimşek |first2=Özgür |last3=Buckmann |first3=Marcus |last4=Gigerenzer |first4=Gerd |date=2022-04-01 |title=Transparent modeling of influenza incidence: Big data or a single data point from psychological theory? |url=https://www.sciencedirect.com/science/article/pii/S0169207020301928 |journal=International Journal of Forecasting |language=en |volume=38 |issue=2 |pages=613–619 |doi=10.1016/j.ijforecast.2020.12.006 |s2cid=234053376 |issn=0169-2070|hdl=21.11116/0000-0007-E16A-4 |hdl-access=free }}</ref> found that Google Flu Trends was outperformed by the ''recency heuristic,'' an instance of so-called "naive" forecasting, where the predicted flu incidence equals the most recently observed flu incidence. For all weeks from March 18, 2007, to August 9, 2015 (the horizon for which Google Flu Trends predictions are available), the mean absolute error of Google Flu Trends was 0.38 and of the recency heuristic 0.20 (both in percentage points; linear regression with a single predictor, the most recently observed flu incidence, had a mean absolute error of also 0.20, and the benchmark of random prediction had 1.80).

One source of problems is that people making flu-related Google searches may know very little about how to diagnose flu; searches for flu or flu symptoms may well be researching disease symptoms that are similar to flu, but are not actually flu.<ref>{{cite web|url=http://siliconangle.com/blog/2014/03/24/google-flu-trends-a-case-of-big-data-gone-bad.|title=Google Flu Trends: A case of Big Data gone bad?|work=SiliconANGLE|date=24 March 2014 }}</ref>  Furthermore, analysis of search terms reportedly tracked by Google, such as "fever" and "cough", as well as effects of changes in their search algorithm over time, have raised concerns about the meaning of its predictions.<ref name="Science"/> In fall 2013, Google began attempting to compensate for increases in searches due to prominence of flu in the news, which was found to have previously skewed results.<ref>{{cite news|agency=NPR|url=https://www.npr.org/templates/transcript/transcript.php?storyId=289802934|title=Google's Flu Tracker Suffers From Sniffles|date=2014-03-13|author=Richard Harris|access-date=2019-12-19|archive-url=https://web.archive.org/web/20190319224139/https://www.npr.org/templates/transcript/transcript.php?storyId=289802934|archive-date=2019-03-19|url-status=live}}</ref>  However, one analysis concluded that "by combining GFT and lagged CDC data, as well as dynamically recalibrating GFT, we can substantially improve on the performance of GFT or the CDC alone."<ref name="Science"/> A later study also demonstrates that Google search data can indeed be used to improve estimates, reducing the errors seen in a model using CDC data alone by up to 52.7 per cent.<ref name="Royal Society Open Science">{{cite journal |last1 = Preis|first1 = Tobias|last2 = Moat|first2 = Helen Susannah|date = 29 October 2014|title = Adaptive nowcasting of influenza outbreaks using Google searches|journal = Royal Society Open Science|volume = 1|issue = 2|pages = 140095|doi = 10.1098/rsos.140095|pmc = 4448892|pmid = 26064532|bibcode = 2014RSOS....140095P}}</ref>

By re-assessing the original GFT model, researchers uncovered that the model was aggregating queries about different health conditions, something that could lead to an over-prediction of ILI rates; in the same work, a series of more advanced linear and nonlinear better-performing approaches to ILI modelling have been proposed.<ref name=NSR>{{cite journal |last1=Lampos |first1=Vasileios |last2=Miller |first2=Andrew C. |last3=Crossan |first3=Steve| last4=Stefansen |first4=Christian |date=3 Aug 2015 |title=Advances in nowcasting influenza-like illness rates using search query logs |journal=Scientific Reports |volume=5 |issue=12760|page=12760 | doi=10.1038/srep12760 |pmid=26234783 |pmc=4522652|bibcode=2015NatSR...512760L }}</ref>

However, followup work was able to substantially improve the accuracy of GFT through the use of a [[random forest]] regression model trained on both the incidence of influenza-like illness and the output of the original GFT model.<ref name=kandula>{{cite journal|last1=Kandula|first1=Sasikiran|last2=Shaman|first2=Jeffrey|title=Reappraising the utility of Google Flu Trends|journal=PLOS Computational Biology|doi=10.1371/journal.pcbi.1007258|date=2019-08-02|volume=15 |issue=8 |pages=e1007258 |pmid=31374088 |pmc=6693776 |bibcode=2019PLSCB..15E7258K |doi-access=free }}</ref>

==Related systems==
Similar projects such as the flu-prediction project<ref name=flupredicition>{{cite web |url=http://www.flu-prediction.com |title=Flu prediction project by the University Osnabrück and IBM WATSON |access-date=2019-12-19 |archive-url=https://web.archive.org/web/20190527185256/http://www.flu-prediction.com/ |archive-date=2019-05-27 |url-status=live }}</ref> by the Institute of Cognitive Science at [[Universitat Osnabruck|Universitat Osnabrück]] carry the basic idea forward, by combining [[social media]] data e.g. [[Twitter]] with [[Centers for Disease Control and Prevention|CDC]] data, and structural models that infer the spatial and temporal spreading <ref name=casual>{{cite journal |url=http://www.mitpressjournals.org/doi/full/10.1162/NECO_a_00756#.Vu5zr0eAY4A |title=A statistical framework to infer delay and direction of information flow from measurements of complex systems |doi=10.1162/NECO_a_00756 |volume=27 |journal=Neural Computation |pages=1555–1608 |access-date=2016-03-20 |archive-url=https://web.archive.org/web/20190212230413/https://www.mitpressjournals.org/doi/full/10.1162/NECO_a_00756#.Vu5zr0eAY4A |archive-date=2019-02-12 |url-status=live |year=2015 |author=Schumacher Johannes |issue=8 |pmid=26079751 |s2cid=25156368 }}</ref> of the disease.

==References==
{{reflist|30em}}

==External links==
* {{official website}}

{{Google LLC}}

[[Category:Internet properties established in 2008]]
[[Category:Projects established in 2008]]
[[Category:Discontinued Google services|Flu Trends]]
[[Category:Influenza]]
[[Category:Data analysis software]]
[[Category:Prediction]]
[[Category:Public health and biosurveillance software]]