India Water Portal Data Project

What is the Data Project?

The IWP Data Project aims to understand what water data exists, creating a diverse and vibrant community that will use that data to enhance projects, advocacy and impact around major water sector indicators.

The main audience for the Data Project remains the current audience for India Water Portal - water experts and practitioners, field-level NGOs, government, think tanks, universities, and corporates. The secondary audience for the Data Project are the Open Data Advocates: software developers, designers, journalists, statisticians, researchers, transparency advocates, NGOs, Government and engaged citizens.

Some expected usage scenarios are thus:

• Field NGOs that are large enough to work in a region to implement some strategy –  they have access to data they need, but IWP can add value by providing data from multiple sources. NGOs benefit most from data but don’t have dedicated funds, time and personnel to access it. Aggregating it and making it easier to find can help them form a better strategy, which is what they really need. 
• NGOs that are training-focussed need updated training materials and the capacity to analyse data that is coming in from their project areas and create materials that are accurate. 
• Grassroots level NGOs need data so that they can assess what happening on the ground and spread knowledge around to partners that is accurate and up to date. 
• Donors/Funding NGOs require good baseline and end-line data to assess where intervention is needed and to evaluate their projects. 
• Advocacy organisations need data to understand what is happening on the ground and create evidence based arguments which support more productive dialogue with stakeholders. 

How was the 'Data Project' conceived?

Over the past 5 years, India Water Portal has been largely concentrating on collating content (case studies, reports, policy documents, manuals, news, opinions, advice) on the above stated water issues.  

A positive shift in the attitudes toward data and some positive experiences, made the move towards a concentrated effort on data - hence the poorly named 'Data Project' was conceived. 

Our main work in the past was publishing 100 years’ worth of climate change data (rainfall, temperature, humidity, etc) on IWP. This data set became one of the most popular sections of the portal, with 34,000 hits over the past 3 years. IWP receives more than 12 requests a month for this data. While water practitioners have used this data to predict crop sowing and cultivation patterns and implement rainwater harvesting infrastructure, they have also compared this data with locally collected data or Indian Meteorological Department data. Researchers used this data for climate change modeling and some citizens have even used this data in unimaginable ways. For example, Thej from Janaagraha used this data set to build this application. Yashas from Srishti School of Design has developed music from this data set. Similarly Anand S has used this data to develop a visualisation to figure out where he wants to live in India.

Secondly, in the past 2 online surveys conducted - in June 2009, and December 2010, the top most expressed need were data, statistics and mapping layers.

Lastly, we are buoyed by the global movement towards open data like Data.gov (US) and Data.uk.gov and work by organizations like MySociety.org, Sunlight Foundation and OFN; local initiatives like the Right to Information Act, and the National Data Sharing and Accessibility Policy; some development organisations like World Bank andGapminder opening up development data and building applications; and the multiplicity of free and/or simple visualisation, mapping and statistical tools that have eased the process for citizens to analyze large data sets.

Why the Data Project?

Future water solutions in India require data to determine appropriate interventions and allow for proper evaluation of those interventions. However, the current state of this data does not allow for these two requirements to be met easily for the following reasons:

• Data is fragmented, scattered, and isolated and there are few searchable data sets that are available online. Most publications and sources of data are available only as hard copy publications from government or in PDFs buried in poorly made websites. The existence and procedures to get this information is not well known.
• Collection of data is fragmented between different agencies. The agencies responsible for collection of the 'physical data' are administered by different Ministries, while the user data comes under such diverse classifications as public health and sanitation, irrigation and urban planning.
• Much of the data is not readily accessible even within and between Government Agencies concerned with water resources development, leave aside in the public domain. (Working Group on Water Database Development and Management, page 22).
• There is surprisingly little awareness about what is available and even some of the online freely available data is unknown. Outreach methods are limited so when a new data source comes online, it takes a long time for people to become aware of its existence. 

An important case in point is that the Department of Drinking Water and Sanitation has put a huge amount of information about its work online. For example, it is possible to see the status of water and sanitation for every village and habitation in the country. This ought to be a rich source of feedback to the government on the status of programmes on the ground. It should also be useful for a lot of analysis on the status of water and sanitation in the country. Also, most NGOs would be interested in understanding the supposed status of water and sanitation in their work areas, as claimed by the government. However, the website is badly designed, but the data is quite rich.

• Lack of critiquing or examination of data means that there is very little known about the quality of the data that is available and whether it is collected and reported accurately. 
• Lack of knowledge of the different data sources and convenient platforms for pre-processing and analysis, remain as important barriers. When data is publicly available, it would be very useful to people. Examples include more fine-grained weather data, river flow data and groundwater prospect maps. 

What is the strategy for the Data Project?

The main role of IWP for the Data Project is to act as a catalyst and innovator in the open data community, and build the open water data community in India. While building this community, IWP shall showcase exciting visualisations, analysis and stories so that the community can see the benefits of working with data. Secondary roles include advocating for open data and identifying new grantees for Arghyam.

There are 5 tracks conceived to actualise this strategy:

1. Data Research (Researching water data to understand what data exists and where is it located)
2. Data Normalisation (Normalisation is the process of putting data into open and accessible formats so that it can be analysed and used by multiple communities)
3. Platform Building (Uploading data online onto multiple platforms ranging from the simple to the complex, so that people can use, and share it easily)
4. Community Building (Grow a community in India around data and water by building bridges between different communities {government, journalists, designers, etc.)
5. Analysis/Visualisation/Story-telling (Creating good data stories around water using visualisation and analysis, and placing them in main stream media)

By focusing  on building community and processes through the 5 Track Framework around data from important water sector indicators we can prove that data is valuable to the water sector and its use can reimagine how we deal with water issues in India. The Data Project does not aim to become a large data warehouse or company; rather it strives to create processes and a framework that is community driven and sustainable.

Want to be involved in the Data Project? You can volunteer in your free time! Email Nisha Thompson (nisha@arghyam.org) or Nisreen Ahmed (nisreen@indiawaterportal.org) for more details.