Rieffel travel | studies | digital life | gadgets | work | music

Extracting specific datasets from openstreetmap to use in ArcGIS

Sometimes you might want to use only a specific part of the whole openstreetmap dataset to use the geodata in a GIS. This little tutorial shows how you could extract data from osm, based on a specific attribute and how to import and use this data within ArcGIS 10. The process requires the following prerequisites:

• most recent version of JOSM
• ArcGIS 10 with installed osm extension

The process includes the following steps:

1. Creating a query string for Openstreetmap’s xapi
2. Downloading the dataset
3. Importing the data in ArcGIS

If you plan to use the openstreetmap data otherwise, you can of course skip the last steps. The following instructions are exemplified by downloading a dataset for the major highways for the area of Belo Horizonte, Brazil, but if modified should work for every other topic and region as well. Please be aware, that this instructions are not suitable to download data for large areas, this will be covered in a separate blog post, since this is a bit more advanced.

1 Creating a query string for Openstreetmap’s xapi

"The OSM Extended API (or xapi, pronounced zappy) is a read-only API, based on a modified version of the standard API that provides enhanced search and querying capabilities. It implements the standard map request and a number of additional ways of querying OSM data by tag values. Xapi uses a REST style interface with X-path flavouring" (more).

If you have some experience with the openstreetmap API, you might just create the query string manually, using the parameters of the API. An easier way to create the query string is offered by the web frontend uixapi. You just need to activate the "Search by area" checkbox and enter the coordinates of your desired area (bounding box) into the respective fields. If you don´t know the coordinates, you can extract them from the openstreetmap.org slippy map in simple steps:

1. Go to openstreetmap.org
2. Center the map view on your desired area
3. Click on the "export" tab
4. Transfer the values for the bounding box to the respective fields of the uixapi

You will already see the link on the bottom of the website changing. In the end, this link is your query string; you need to query the xapi. In a next step, you select your desired key/value pair from the openstreetmap wiki and enter it in the respective field on top of the page. In this example it would be highway=motorway since we want to query for all major highways in a certain area. This completes the necessary steps on this website. To run a first test, you could now simply click the link on the bottom of the page. If it returns some kind of xml values to you, your query works fine and you can go on. If you encounter an error like "HTTP Status 500 - Maximum bounding box area is 10.0 square degrees" your selected area might be too big. Adjust your bounding box then and retry.
In case you want to combine multiple attributes into one query (e.g. highway=motorway and highway=motorway_link) read up in the xapi documentation. You might need to modify your link manually afterwards.

2 Downloading the dataset

The easiest way to download your selected dataset would be via the openstreetmap java editor josm. Get the latest version here and start it. Afterwards, select "File" -->"Open from Address" and paste your link created in step one. The link to select all motorways in the area of Belo Horizonte for example looks like this:

http://jxapi.openstreetmap.org/xapi/api/0.6/*[highway=motorway][bbox=-44.5,-20.47,-43.46,-19.36]

This will download the data you selected earlier into josm and visualize it as well. Now you can save your dataset with a right click on the data layer, then select "save as" and choose "osm server data". Now you have a cut out of a specific region based on specific attributes saved to your machine. You can now use this for many purposes; i will go on and explain the use of it in ArcGIS 10 now.

3 Importing the data in ArcGIS

To import the *.osm files into ArcGIS 10, you need to install the ArcGIS editor for openstreetmap first. Add the OSM Editor tool to your toolbox (right-click, "Add Toolbox"--> Browse to the install path of the ArcGIS editor for openstreetmap and click "open". To import the data, you need a geodatabase, so either use the default one or create a new one using the catalog. Then open the "Load OSM File" tool from the toolbox, enter the required parameters and run it. If it terminates with some kind of undefined error, dare to try again some more times, it still seems to have some errors, which i could neither figure out nor understand, but retrying sometimes seems to solve this problem. Now you have a geodatabase with your osm data in a format that can be understood by ArcGIS. If you examine your data closer, you see that you have points, lines and polygons, you can use and manipulate them however you like in your GIS, use them for calculations or as well just simple mapping.

4 Notes

To download the data from the server you could also use a tool like wget which is also available for windows.
If you want to get data of a bigger area than the maximum allowed area of xapi, you need to download the respective osm data, either as an extract from geofabrik.de or the whole planet dump, and then use a tool like Osmosis to extract the data you need. I will write more about this in an upcoming blog post.
if you encounter any errors, have questions or just want to say something, I am looking forward to hear from you in the comment section