Open Access is not Open Data
Not all 'Open' initiatives are equal
In the Open Data community I have noticed an occasional blurring of two very different concepts; Open Data and Open Access (in the sense of 'freely available documents'). This can cause problems and increase costs for all stakeholders involved (from data availability to FOI burden).
I argue here that we (managers, data scientists, developers) need to strongly differentiate the two, as it has a real impact on the community stakeholders - the individuals, businesses and institutions that consume our data.
In this article I'll set out a case (from a consumer perspective) for each, to provide some insight into how these are two different things for two very different audiences.
What is the difference?
Open Data - Re-use and Remix
Open Data is about making raw data accessible to others, with few restrictions. For example, a spreadsheet (say, a CSV file) that can be quickly opened in Excel, the contents readily analysed and used for whatever purposes you have.
Open Data is published with the intent of others using the data for their own purposes. As such, it's published in a file format that can be opened and analysed, and the licence of the data comes with very few restrictions (typically focusing on attribution to the data source). This is what makes it "open" - it's open to reuse.
The primary audience of Open Data is developers, automated systems, data analysts, data scientists, companies and organisations.
Open Access - Documents and Communication
Open Access is about communicating information to people, for example, in reports with graphs, charts and explanations of research done, decisions that have been made or how certain goals are being met.
Open Access is published with the intent of whoever is interested being able to read a document and possibly draw their own conclusions. It is not designed to have its raw information easily used in another context by the reader - say extracting prices from a chart inside the document.
In general, it would be difficult for a programmer to reliably extract data from a PDF containing a mix of text and various graphs and charts. Furthermore, the licence on that extracted data is often unclear, making it legally risky to use.
Why does it matter?
Opportunity cost. If an institution commits a lot of time and effort to producing Open Access documents, in practice they're less likely to commit even more resources to an Open Data distribution of the "same" data. Not always the case, but it's budgetary pressure on what some would consider to be "a box that has already been ticked".
For public institutions, this tends to drive an increase in FOI (freedom of information) requests, which are much more expensive due to being processed case-by-case.
For those of us working in data science, communicating this difference clearly is the difference between;
- closed vs open ecosystems,
- automation vs tedious manual data extraction,
- licensing ambiguity (with potential legal risks) vs clarity
For developers, it's the difference between a quick and easy integration vs being unable to implement features because while the information is clearly available to someone it is not available to you.
What's the use case?
Apps (Open Data)
An app developer wanting to show disability access to various public buildings and spaces would need to access Open Data to integrate into their app services.
Doing research (Open Data)
A data scientist researching childhood asthma would need to use raw data readings from sensors outside schools. A document summarising pollution levels may obscure specific elements of the data, like time of day variations.
Making decisions (Open Access)
The CEO of a company looking to open a new carbon-neutral manufacturing facility might rely on Open Access reports about per-area emissions, local recycling initiatives and other such relevant data.
Providing insight (Open Access)
A parent might review several Ofsted reports about the local schools to pick the best for their child.
Cost
When you publish Open Data the costs are strongly front-loaded, typically with a lower, persistent upkeep cost. You build a system (maybe an API, maybe a simple CMS site) and create an automated pipeline to extract data from existing internal data sources to be published into the Open Data portal. The costs after that tend to be technical maintenance and infrastructure (depending on the scale of the data and number of users, this can range from "negligible" to "significant").
When you publish Open Access the costs are repeated every time you generate a new report. Authors, editors, data scientists, management and more are typically involved in every published document - and every new document requires their contributions again. In short, it's an expensive recurring cost - and because it's expensive, it's done infrequently.
To the data consumers, there are costs to Open Access too; For those brave or desperate enough to try to scavenge data out of HTML pages, PDFs, DOCX (and other document formats) the cost is in time, licence risk and accuracy.
Not wanting to detract from the main point of this article, you don't have to use Hypermass to publish data cheaply, we are just one option.
With that said, we built Hypermass.io because we got tired of this specific problem. We wanted a way to make the 'Open Data' part so trivial that more people would do it.
Which should you do?
Consider what you want to achieve;
- are you directly reaching out to highly specialised stakeholders who you know will want specific questions answered?
- Is the goal communication of the facts to a key group of people?
- or to provide insight into the reasoning behind a decision process?
... if so, you likely want to compile Open Access documents specialised for your audience.
If, however, you're considering a more varied audience;
- you don't know how the data will be used, but you know there is a public interest in it
- or you can see the data being useful for companies, apps, researchers
- or if people are showing interest in certain data
... then Open Data may better match the interests of stakeholders.
Often the answer is a combination of both - in recognition of the different needs of a varied set of data consumers. I'd recommend taking a data-first approach since this enables others to build on the data and provides an accessible foundation that can be cited from Open Access documents.