Data

From Opendataday

Contents 1 Tools for Data Scraping and Sharing 2 Global or Regional Datasets 2.1 Catalogs 2.2 Datasets 2.3 Multi-domain 3 UK 3.1 Catalogs 3.2 Datasets 4 Netherlands (Dutch) datasets 4.1 Catalogs 5 Belgian Datasets 6 Canada Datasets 6.1 Catalogs 6.2 Official datasets 6.3 Unofficial datasets 7 United States Datasets 8 Australia Datasets 9 New Zealand Datasets 10 Spain Datasets 11 Brazilian Data Sources 11.1 Federal Government 11.2 States 12 Indian Datasets 13 404s

Know of a open data set that you think people might want to use in an app? Add it here. Also, know of a data set that you think, if scraped, might be helpful to write an app? Add it here too!

DataCatalogs.org and The DataHub link to or host many of the datasets listed below. In particular, the DataHub there is a dedicated Open Data Day dataset group.

Also note existing list of open data catalogs (government and otherwise): http://datacatalogs.org/

Tools for Data Scraping and Sharing

• theDataHub - Open Data Day Group on theDataHub
• Buzz Data
• Scraper Wiki
• Junar Searching for #OpenDataDay you will find a bunch of data collected for the occasion. You can also collect your own.

Global or Regional Datasets

Catalogs

Most of the following are already listed in http://datacatalogs.org.

• the DataHub - data hub and data catalog where anyone can store and register datasets
• World Bank data catalog - World Bank DataHub Dataset
• PublicData.eu - public data from across Europe
• Guardian list of world government datasets
• International Aid Transparency Initiative (IATI) data - international development / aid data
  • More information on IATI data from Publish What You Fund
• World Bank's financial data, with tools to slice, dice, and visualize - World Bank Finances DataHub dataset

Datasets

• Aiddata's development finance project database and their API - AidData DataHub dataset
• OpenStreetMap (maps) - OpenStreetMap DataHub dataset
• OECD (economic data)
• OpenEI.org (Open Energy Information) has a library of energy information and data, also available by SPARQL, Ask and some REST services
• UN data api project
• Global Aid Flows
• Geo-referenced Catalog of all World Bank Projects for Africa and Latin America

Multi-domain

Datasets already collated from multiple sources:

• dbpedia - DBPedia DataHub dataset
• Freebase Freebase DataHub Dataset

UK

See http://thedatahub.org/dataset?q=country.uk+country-uk

Catalogs

• data.gov.uk
• London data catalog
• Greater Manchester data hub and catalog
• (Deprecated - replaced by http://datacatalogs.org) OpenGovernmentData catalogue spreadsheets overview and spreadsheets

Datasets

• Police API
• UK Charities - Open Charities DataHub Dataset
• recent Jobcentre Plus vacancies - scraped data
• Openly Local - OpenlyLocal DataHub dataset
  • UK Planning Alerts - now shut down but may become part of OpenlyLocal - Planning Alerts DataHub dataset
• UK Freedom of Information requests & responses
• The Gavel Scraper - live data from every UK Crown Court
• UK Towns & Cities
  • Lichfield DC
  • Warwickshire CC
  • Kent CC

Netherlands (Dutch) datasets

Catalogs

• http://www.overheid.nl/opendata/
• http://www.opentaal.org/
• http://www.dans.knaw.nl/
• http://nl.ckan.net/
• http://data.weetmeer.nl/
• http://data.3tu.nl/repository/
• http://www.openeindhoven.nl
• http://eindhoven.buurtmonitor.nl/

Belgian Datasets

• Train information in Belgium
• A site which contains a lot of open data for Belgium

Canada Datasets

Catalogs

The Canada group on DataCatalogs.org has a list of all open data hubs in Canada.

Official datasets

• NEPTUNE Canada (free login required) Ocean science data

Unofficial datasets

• BC Elections Political Contributions
• Various Federal Elections Contributions
• WinnipegElection.ca Datasets
• Water and Environmental Hub - open water and environmental data available for download and through RESTful API

United States Datasets

The United States group on DataCatalogs.org has a list of all open data hubs in the United States.

The Sunlight Foundation offers public datasets for each of its projects, including:

• Transparency Data, which sources OpenSecrets.org, FollowTheMoney.org, POGO.org
• Sunlight Labs APIs
• Open States
• Subsidyscope - Subsidy Scope DataHub dataset
• Clearspending
• House Staff Directory
• House Expenditure Reports
• Campaign Ad Monitor

Additional data sources:

• GovTrack.us - GovTrack DataHub entry
• DemocracyMap.org
• Lobbying Disclosure Filings (XML)
• Lobbying Contribution Filings (XML)
• OpenEI.org (Open Energy Information) - energy data with web services, some datasets are specific to the U.S., such as the utilities and incentives information
• NREL (National Renewable Energy Laboratory) - solar, transportation and other energy web services

Australia Datasets

• The Australia group on DataCatalogs.org has a list of all open data hubs in Australia.
• Scrapers tagged Australia on ScraperWiki.

New Zealand Datasets

• data.govt.nz - all datasets in http://data.govt.nz have been scraped into http://thedatahub.org/

Spain Datasets

The Spain group on DataCatalogs.org has a list of all open data hubs in Spain.

Brazilian Data Sources

• Brazilian community datahub

Federal Government

• http://www.ipeadata.gov.br
• http://www.portaltransparencia.gov.br
• http://tse.gov.br
• http://www.datasus.gov.br
• http://www.receita.fazenda.gov.br
• http://www.inep.gov.br
• http://www.ibge.gov.br
• COI - Presidencia da Republica

States

• Ceará: API do Tribunal de Contas dos Municípios do Ceará
• Espírito Santo: Portal de transparência do Espirito Santo

Indian Datasets

• https://finances.worldbank.org/facet/countries/India
• http://data.worldbank.org/country/india

404s

Sites listed here but which 404.

• http://www.opendatani.info/ (Northern Ireland)
• ReliefWeb (disasters)
• TheGraph.net