Oxford, UK
From Opendataday
Some of what happened on the day: http://opendatahackathonoxford.posterous.com/
An Oxford Open Data Hack Day was held on December 4th, hosted at the offices of White October:
- 3 The Gallery (Unit 3 is just down the side of the building)
- Marston Street,
- just off the Cowley Road,
- East Oxford
- Open Street Map: http://osm.org/go/eutD6HPKx-
- Street View: http://bit.ly/bzx90z
- 15 minutes by bus from Oxford Railway Station / Short walk from 'The Plain' stop on the Oxford Tube bus from London.
Questions/comments to tim@practicalparticipation.co.uk.
The Oxford Team
If you have any questions, problems or special requests then you can contact any of these guys:
- Tim Davies (@timdavies)
- Dave Fletcher (@dafletcher)
- Ed Lucas (@mightjustwork)
- Kevin Carmody (@skinofstars)
- Sebastian Motraghi (@seb_m)
- Ed Wong (@platform3)
Agenda
- 10.00 - Introduction & Finding Teams (some people will be around earlier if you just can't wait to get started)
- 10.30 - Planning, Research and Initial Project Development
- 13.00 - Lunch
- 13.30 - Project Development - working in teams, and pulling in extra support where you need it
- 16.30 - Demos - It could be a working website; a mock-up; a prototype; a worked-up idea; a statistical finding - whatever you think shows what can be done with public data.
We are going to be joining the Videoring at 1630 GMT
Projects
Projects worked on
- Robin Hood Tax viz - Demo site Exploring the impact of Robin Hood taxes - also Geodata Demo of RHT Data.
- Arts funding viz - Demo Site Tags demo site Stacks of cash demo site Using data about Arts Council funding, this team is visualizing where and to who the funding has been awarded.
- Twitter geo sentiment - AKA Twittycity Who is happy (or otherwise) in different UK cities using Twitter geo-data.
- World Bank data - Using World Bank data so that someone can enter a location and find Word Bank projects or other projects if there is data for it, near them.
- Local transport - Tracking the location of Oxford buses using the data from Oxontime.
- dbpedia/Ordnace Survey Open Data - Map Demo site Using @gothwin's work for Ordnance survey and dbpedia to geolocate you with your postcode and finds something interesting nearby from dbpedia..."It makes Blue Plaques for you wherever you are"
- CKAN - Demo site Using the data.gov CKAN API to see the top 15 tags to see the relationships between the packages.
Project Ideas
Got an idea for a project? Jot ideas below:
- Open Data for Campaigning - exploring how campaigning organisations can make use of open government data.
- Improving Oxfordshire Scrapers - check out how well the scrapers providing data for OpenlyLocal.com and PlanningAlerts.com are working for Oxfordshire Councils and spend time improving them. (Planning alerts uses ScraperWiki to gather data). See This How To for details on creating a scraper - with an example scraping the Oxford City Council website.
It might be interesting as well to explore creating scrapers for committee meeting minutes etc such as these. OpenlyLocal.com is also not grabbing much Oxfordshire County Council at the moment. See [1].
- Journalism Hacks - exploring the possibilities to use data to generate local news stories.
- BikeSafe - can we get data on cycle thefts? Could it be used to find the best and worst places to leave a bike?
- Sentiment Analysis in Oxford - use open data from sources such as Thames Valley Police, local news sites e.g. Oxford Mail as well as data mined from Twitter, Facebook Graph API etc. This could provide a rich set of data upon which sentiment analysis could be performed. Visualisations could be built from this data e.g. general happiness levels based on location within Oxford.
- Visualising the data - Important to ensure that getting the data is only step: analysing and visualising it an appropriate way is key to gaining insight.
- Which bus should i get - so if you have to change the bus, in town, which company would be better
Interested Participants
Please use the registration form so we know your skills and interests, allowing us to tailor the event to the skills available... It also means we know how much pizza and beer to buy :)
There is also a Lanyrd.com event page for the day
You can also add your name below to let others know you're going.
Name / Contact / Preferred-role / Any particular project or issue you are interested in working on
- Tim Davies / @timdavies / data-monkey / Interested in working on some local data... not sure what yet
- Ed Lucas / @mightjustwork / developer / Put me to work...
- Richard Fairhurst / @richardf / geo-developer / Sustainable transport - rail and cycling in particular
- Kevin Carmody / @skinofstars / webdev, tea & snacks runner / Interested in local data and community engagement
- Alexis Metaireau (@ametaireau) / python, django dev / transports - cycle, interested by GIS data.
- Sébastien bocahu (zecrazytux)
- Martha LaGess (@LaMaStudio) Physical/ virtual architect wants to make it easy for anyone to access and use environmental and social data for individual and organizational purposes. Based in London & wondering whether we could arrange virtual link that could create interchange between the now-forming London Hackathon group and the Oxford Hackathon group, before the event, on the day, or both? Love the idea of creating data-sets lists as part of this event. Consider it crucial to bring in some visualization and GIS people.
- Nick Morgan / @skilldrick / developer / If I can make it I'd be happy to help with whatever server- or client-side dev work needs doing.
- Pablo de la Peña / @hellopablo / Ninja Geek: dev - UX - Interaction Design / Just wanna get my hands dirty with code, photoshop and user experience.
- Peter West / @peterjwest / developer, UI designer / Up for anything! Experience with genetic programming if that's any help.
- George Hickman / @ghickman / Developer / Happy to do whatever!
- Andy Cotgreave / @acotgreave / visualisation / Transform the data into rich visualisations. Into cycling and transport data
- Alexander Dutton / @alexsdutton / developer at OUCS, RDF and data wrangling enthusiast
- Dan Goodwin / @bouncingdan / observer, runner, front end dev, UX, UI / Helping out, taking photos, reporting, advice and help with UI and front endy stuff
- Ed Wong / @platform3 / Geek herding, tea making , pizza deliveries
- Zarino Zappia / @zarino / Designer, Coder and Internetologist
- Arnaud Bos / @arnaud_bos / Brookes student, java, python, flex, android dev / busses, cycle, transports
- Robert (Jamie) Munro / @rjmunro / Python & PHP coder / I've written an Oxontime scraper in the past that turned the map into a kml feed. Would like to spend some time and cleaning the code, then put a REST API on it for other projects to play with.
- Al Power / @alpower / Front end dev, java, UX, photographer, front endy stuff, helper, tea maker. Looking to find out more on dataViz!
- David Pidsley / @davidpidsley / developer, java, gwt, gae / open charity data
Datasets of Interest
Add datasets that might be of interest below. We'd also love to see some Oxford/Oxfordshire focussed sets for hackery
Open Knowledge Foundation
Not for profit organisation dedicated to promoting open data http://okfn.org/ also see the CKAN catalogue at http://www.ckan.net/ which is behind data.gov.uk and other open data sets.
Data.gov.uk
Any national datasets at http://data.gov.uk which could have Oxford/Oxfordshire data pulled out of them (e.g. EduBase)
Police API
http://policeapi.rkh.co.uk/ - (need to apply for API Key in advance)
Charities data
http://opencharities.org/ - full dataset contains details of all UK Charities; could filter for local charities.
Jobcentre Plus Databases
Scraped list of all recent vacancies.
Isis Flood and River Flow
Some historical data on Isis Flags is available from the Isis Level site.
http://www.atm.ox.ac.uk/rowing/river.html
Openly Local
Oxford City Council on Openly Local and other local councils also - provides list of Councillors, Committees etc. (There may be a useful job to be done improving the local scrapers for local council sites...)
http://openlylocal.com/councils/350-Oxford-City-Council
http://openlylocal.com/councils
Planing Alerts
http://www.planningalerts.com/
Details of local planning applications (Example query) - also may be a need to check and improve other local scrapers.
Freedom of Information
Freedom of Information requests and responses. Incomplete and often in non machine readable formats, but some very interesting stuff none the less. They do have an Oxford(shire) section.
http://www.whatdotheyknow.com/
Oxford Mail
Not an API, but could be scrapable
http://www.oxfordmail.co.uk/li/
Guardian
http://www.guardian.co.uk/open-platform
British Social Attitudes
Every year the British Social Attitudes survey asks around 3000 people what it's like to live in Britain and how they think Britain is run. The survey tracks people's changing social, political and moral attitudes and informs the development of public policy.
http://www.natcen.ac.uk/study/british-social-attitudes-26th-report
Understanding Society
Understanding Society is a world leading study of the socio-economic circumstances and attitudes of 100,000 individuals in 40,000 British households. It is funded by the Economic and Social Research Council (ESRC) and run by the Institute for Social and Economic Research (ISER). The study allows for deeper analysis of a wide range of sections of the population as they respond to regional, national and international change.
http://www.understandingsociety.org.uk/
Citizenship Survey Since 2001
The Citizenship Survey (formerly known as the Home Office Citizenship Survey, or HOCS) has been commissioned every two years. Approximately 10,000 adults in England and Wales (plus an additional boost sample of 5,000 adults from minority ethnic groups) are asked questions covering a wide range of issues, including race equality, faith, feelings about their community, volunteering and participation. In 2003 there were supplementary booster samples of children (eight and nine-year-olds), young people (10 to 15-year-olds), and 20 local areas.
Defra’s Survey
- of Public Attitudes and Behaviours Toward the Environment
It gives a representative picture of what people in England think, and how they behave, across a range of issues relevant to the environment, including transport and waste recycling. It informs the segmentation model which looks at how environmental attitudes, values, current behaviours and motivations and barriers are packaged together for defined segments of the population. It has implications for policy and communications development as well as identifying areas for further research.
http://www.defra.gov.uk/evidence/statistics/environment/pubatt/download/pubattsum2007.pdf
NatCen
They carry out and analyse research studies in the fields of social and public policy including extensive research among members of the public.
http://www.natcen.ac.uk/about-us
Survey Resource Network
The Survey Resources Network (SRN) is a new service funded by the Economic and Social Research Council (ESRC) to co-ordinate and develop ESRC activities related to survey research methods.
The run similar sessions to your ‘open hack day’ but call them workshops and placements whereby people attend a specific workshop on a particular data set/survey and then are offered placements with the individual survey ‘providers’ A list of their timetable and who runs the survey is listed here:
http://www.surveynet.ac.uk/ssp/workshops/scheduled.asp - this may come in handy when looking for project leads, designers and developers.
Government Spending
The Guardian have released data on government spending:
http://www.guardian.co.uk/news/datablog/2010/nov/19/government-spending-data#
Visualisation competitions
Mozilla have a nicely timed competition to vizualize their usage data. This would be fun to enter and share the prize with! https://testpilot.mozillalabs.com/testcases/a-week-life-2/aggregated-data.html (added by Andy Cotgreave)
Visualizing.org has a nice medical challenge with a great prize - attendand at TEDMED.
University datasets (incl. University of Oxford)
The Higher Education Statistics Agency make available summary stats. They also provide insitution level XLS files.
The Computing Services have made OxPoints available under a CC0 license. OxPoints is an RDF dataset describing the department and colleges of the University, the buildings they occupy, and the political and geospatial relationships between them.
There is an interactive view of the Student Stats on the University's website. Data could be scraped via the Download data button, I guess. More detailed student statistics for University of Oxford are found in Tableau files. The downloadable files are just zips - they could be unpacked and the data extracted. The underlying data will be harder to extract.
Norrington data (which ranks colleges) can be downloaded.
The University of Oxford Careers service provides data on Graduate Destinations.
Useful tools
- Scraper Wiki - code frameworks and hosting for writing web scrapers.
- SIMILE Exhibit - feed it nice json (or even Google Spreadsheets) and quickly explore what a dataset contains; mix with datapress for even easier interface creation.
- Google Refine (formerly Freebase Gridworks) - powerful data-cleaning tool (see Jeni Tennison's notes on using Gridworks to create linked data)
- Google Fusion Tables - (a) tool for mashing together, geocoding and visualising large datasets; (b) a useful source of data as well.
- Tableau Public - a data viz product: this is the free version of the full app for bloggers/public data