How can I create automated data downloads?


Ordnance Survey is one of the oldest data organizations in the world. We have been collecting information about Britain since 1791, organizing, analyzing and disseminating maps and other information. As information technology has evolved from charts and graphs and pen and paper calculations to databases, vector graphics and sophisticated machine learning algorithms, OS has been a leader in adopting and innovating new ways to capture, store, process and share valuable location information.

borderline-base-zoomstack-newcastle-small

As the scale of data increases, so does its potential usefulness – and the complexity of its management. Understanding a small table of numbers is easy, less so for a complex database with dozens of tables and millions of rows. With code however, it has never been easier for programmers to retrieve and manipulate large volumes of data and extract and understand the valuable information they contain.

As all data scientists and web developers know, well-structured datasets are hard to find and even harder to create. Data management is a major task and is often overlooked by those less familiar with the data science process. For algorithms to extract accurate and meaningful information from a dataset, especially using powerful, freely available analysis libraries like Python’s pandas, the data must be consistently structured and clean.

With the OS Maps, OS Vector Tile, and OS Features APIs, users authenticate their request with an API key created on their OS Data Hub account where they can dynamically retrieve the data they need when they need it. This is extremely useful in web maps, data science workflows, and desktop GIS applications. These APIs allow developers and spatial analysts to create and customize interactive maps and retrieve location data with rich attribution – information about a useful feature for visualization and analysis.

The OS Downloads API makes a number of OS OpenData datasets available for automated download over the Internet. This new service makes collecting useful spatial data from Ordnance Survey as easy as pasting a URL or sending an HTTP GET request. The OS Downloads API documentation contains sample code, and we’ve created a tutorial to help JavaScript developers retrieve data programmatically with NodeJS.

The OS Data Hub APIs are the result of a major effort by OS to make our national geographic database easier to access than ever. A wide range of users in Britain, from government officials to NGO workers, academic researchers, entrepreneurs and large companies, could improve the quality and efficiency of their work with the insights they could derive from the data. of the operating system. We work hard to make sure the path from data to decision making is as short as possible. One way we do this is through automated data downloads.

We’ve written a tutorial to help you get started with automated data downloads from the operating system’s download API using NodeJS – find it here.


Sam D. Gomez