wiki:GIS/OpenStreetMap

Context Navigation

Version 9 (modified by Fran Boon, 11 years ago) ( diff )
osm2pgsql is now on GitHub

OpenStreetMap

Wikipedia of Maps

http://openstreetmap.org

Base Map

We have out-of-the-box the ability to use OpenStreetMap Tiles as base layer.

This can include local OSM sites (OSM Taiwan is included as an example)

Local Tile Server

If you need to run offline then you need to set up your own Tile Server:

OpenStreetMap data can be stored in a PostGIS database & rendered using Mapnik.

To support Key/Value pairs:

apt-get install postgresql-contrib-9.3
su postgres
psql -d gis -f /usr/share/postgresql/9.3/contrib/hstore.sql

To support Planet diff files:

su postgres
psql gis < /usr/share/postgresql/9.3/contrib/_int.sql

Osm2pgsql

http://wiki.openstreetmap.org/wiki/Osm2pgsql

It is recommended to use the latest version from source rather than an out of date package:

apt-get install build-essential libxml2-dev libgeos-dev libpq-dev libbz2-dev proj autoconf subversion libtool
git clone https://github.com/openstreetmap/osm2pgsql
Optional (patch updated from original here): http://www.mail-archive.com/dev@openstreetmap.org/msg12768.html
#wget http://eden.sahanafoundation.org/raw-attachment/wiki/InstallationGuidelinesGISDataLinux/osm2pgsql_centroid.patch
cd osm2pgsql
#patch -p0 < ../osm2pgsql_centroid.patch
./autogen.sh
./configure
sed -i 's/-g -O2/-O2 -march=native -fomit-frame-pointer/' Makefile
make
make install
# Fix needed for SVN version 0.70.5
cp default.style /usr/local/share

Mapnik

http://wiki.openstreetmap.org/wiki/Mapnik
http://mike.teczno.com/notes/mapnik.html (Advanced Tips)

0.7.1 is packaged for Squeeze:

apt-get install python-mapnik unzip

Download the OpenStreetMap extensions for Mapnik:

cd
svn export http://svn.openstreetmap.org/applications/rendering/mapnik

Download/decompress Coastlines:

cd
wget http://tile.openstreetmap.org/world_boundaries-spherical.tgz
wget http://tile.openstreetmap.org/processed_p.tar.bz2
wget http://tile.openstreetmap.org/shoreline_300.tar.bz2
wget http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/10m-populated-places.zip
wget http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/110m/cultural/110m-admin-0-boundary-lines.zip
cd ~/mapnik
tar zxvf ~/world_boundaries-spherical.tgz
tar jxvf ~/processed_p.tar.bz2 -C world_boundaries
tar jxvf ~/shoreline_300.tar.bz2 -C world_boundaries
unzip ~/10m-populated-places.zip -d world_boundaries
unzip ~/110m-admin-0-boundary-lines.zip -d world_boundaries

Disable autovacuum:

vim /etc/postgresql/8.4/main/postgresql.conf
#autovacuum = on
/etc/init.d/postgresql restart

Download/Import area of interest, e.g.:

#wget http://downloads.cloudmade.com/north_america/haiti/haiti.osm.bz2
#su postgres
#osm2pgsql -s -d gis haiti.osm.bz2
wget http://labs.geofabrik.de/haiti/2010-11-24-14-43.osm.bz2
su postgres
osm2pgsql -s -d gis 2010-11-24-14-43.osm.bz2

Re-enable autovacuum:

vim /etc/postgresql/8.4/main/postgresql.conf
autovacuum = on
/etc/init.d/postgresql restart

Setup XML file:

cd ~/mapnik
./generate_xml.py --accept-none --dbname gis --symbols ./symbols/ --world_boundaries ./world_boundaries/

Render tiles:

cd ~/mapnik
vim generate_tiles.py
render_tiles(bbox, mapfile, tile_dir, 0, 8, "World")
#minZoom = 10
#maxZoom = 16
#bbox = (-2, 50.0,1.0,52.0)
#render_tiles(bbox, mapfile, tile_dir, minZoom, maxZoom)
# Haiti
bbox = (-74.668, 17.884, -71.659, 20.232)
render_tiles(bbox, mapfile, tile_dir, 9, 17, "Haiti")
# Muenchen
# comment all under here

mkdir /var/www/tiles
chown postgres /var/www/tiles
ln -s /var/www/tiles
su postgres
MAPNIK_MAP_FILE="osm.xml" MAPNIK_TILE_DIR="tiles/" ./generate_tiles.py

Contours

http://wiki.openstreetmap.org/wiki/Contours#The_PostGIS_approach

apt-get install gdal-bin

SRTM3 data:

mkdir ~/mapnik/srtm
cd ~/mapnik/srtm
wget http://mapnik-utils.googlecode.com/svn@170/sandbox/testing/hillshading/srtm_generate_hdr.sh 
chmod +x srtm_generate_hdr.sh 
cp srtm_generate_hdr.sh /usr/local/bin
vim process_srtm3.sh
#!/bin/bash 
PREP_TABLE="1" 
for X in *.hgt.zip; do 
    yes | srtm_generate_hdr.sh $X 
    rm -f "${X%%.zip}" 

    # Import 10m contours 
    rm -f "${X%%.hgt.zip}.shp" "${X%%.hgt.zip}.shx" "${X%%.hgt.zip}.dbf" 
    gdal_contour -i 10 -snodata 32767 -a height "${X%%.hgt.zip}.tif" "${X%%.hgt.zip}.shp" 
    [ "$PREP_TABLE" ] && shp2pgsql -p -I -g way "${X%%.hgt.zip}" contours | psql -q gis 
    shp2pgsql -a -g way "${X%%.hgt.zip}" contours | psql -q gis 

    rm -f "${X%%.hgt.zip}.shp" "${X%%.hgt.zip}.shx" "${X%%.hgt.zip}.dbf" 
    rm -f "${X%%.hgt.zip}.bil" 
    rm -f "${X%%.hgt.zip}.hdr" 
    rm -f "${X%%.hgt.zip}.prj" 
    rm -f "${X%%.hgt.zip}.tif" 
    unset PREP_TABLE 
done

# Haiti
wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N17W072.hgt.zip
wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N18W072.hgt.zip
wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N18W073.hgt.zip
wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N18W074.hgt.zip
wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N18W075.hgt.zip
wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N19W072.hgt.zip
wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N19W073.hgt.zip
wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N19W074.hgt.zip
wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N19W075.hgt.zip
wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N20W073.hgt.zip
wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N20W074.hgt.zip
wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N20W075.hgt.zip

chown postgres .
su postgres
sh process_srtm3.sh

Since we are non-commercial, we should be able to use the SRTM4.1 data from CGIAR, however the Shapefile production crashed when it reached 2Gb.
Example for Haiti:

cd ~/mapnik/srtm
wget http://srtm.csi.cgiar.org/SRT-ZIP/SRTM_V41/SRTM_Data_GeoTiff/srtm_22_09.zip
unzip srtm_22_09.zip
gdal_contour -i 10 -snodata 32767 -a height "srtm_22_09.tif" "srtm_22_09.shp"
su postgres
shp2pgsql -p -I -g way "srtm_22_09" contours | psql -q gis
shp2pgsql -a -g way "srtm_22_09" contours | psql -q gis

Configure Mapnik:

vim ~/mapnik/osm.xml
# Under line &layer-shapefiles;
&layer-contours;
vim ~/mapnik/inc/layer-contours.xml.inc
<Style name="contours10">
        <Rule>
                &maxscale_zoom14;
                &minscale_zoom17;
                <LineSymbolizer>
                        <CssParameter name="stroke">#9cb197</CssParameter>
                        <CssParameter name="stroke-width">0.5</CssParameter>
                </LineSymbolizer>
        </Rule>
</Style>
<Style name="contours50">
        <Rule>
                &maxscale_zoom14;
                &minscale_zoom17;
                <LineSymbolizer>
                        <CssParameter name="stroke">#9cb197</CssParameter>
                        <CssParameter name="stroke-width">0.6</CssParameter>
                </LineSymbolizer>
        </Rule>
        <Rule>
                &maxscale_zoom12;
                &minscale_zoom13;
                <LineSymbolizer>
                        <CssParameter name="stroke">#747b90</CssParameter>
                        <CssParameter name="stroke-width">0.6</CssParameter>
                </LineSymbolizer>
        </Rule>
</Style>
<Style name="contours100">
        <Rule>
                &maxscale_zoom14;
                &minscale_zoom17;
                <LineSymbolizer>
                        <CssParameter name="stroke">#9cb197</CssParameter>
                        <CssParameter name="stroke-width">0.7</CssParameter>
                </LineSymbolizer>
        </Rule>
        <Rule>
                &maxscale_zoom12;
                &minscale_zoom13;
                <LineSymbolizer>
                        <CssParameter name="stroke">#747b90</CssParameter>
                        <CssParameter name="stroke-width">0.7</CssParameter>
                </LineSymbolizer>
        </Rule>
        <Rule>
                &maxscale_zoom10;
                &minscale_zoom11;
                <LineSymbolizer>
                        <CssParameter name="stroke">#855d62</CssParameter>
                        <CssParameter name="stroke-width">0.7</CssParameter>
                </LineSymbolizer>
        </Rule>
</Style>
<Style name="contours-text50">
        <Rule>
                &maxscale_zoom14;
                &minscale_zoom17;
                <TextSymbolizer name="height" face_name="DejaVu Sans Book" size="8" fill="#747b90" halo_radius="1" placement="line" />
        </Rule>
</Style>
<Style name="contours-text100">
        <Rule>
                &maxscale_zoom14;
                &minscale_zoom17;
                <TextSymbolizer name="height" face_name="DejaVu Sans Book" size="8" fill="#747b90" halo_radius="1" placement="line" />
        </Rule>
        <Rule>
                &maxscale_zoom12;
                &minscale_zoom13;
                <TextSymbolizer name="height" face_name="DejaVu Sans Book" size="8" fill="#855d62" halo_radius="1" placement="line" />
        </Rule>
</Style>
<Layer name="srtm_10" status="on" srs="+proj=latlong +datum=WGS84">
        <StyleName>contours10</StyleName>
        <StyleName>contours-text10</StyleName>
        <Datasource>
                <Parameter name="table">(select way,height from contours WHERE height::integer % 10 = 0 AND height::integer % 50 != 0 AND height::integer % 100 != 0) as "contours-10"</Parameter>
                &datasource-settings;
        </Datasource>
</Layer>
<Layer name="srtm_50" status="on" srs="+proj=latlong +datum=WGS84">
        <StyleName>contours50</StyleName>
        <StyleName>contours-text50</StyleName>
        <Datasource>
                <Parameter name="table">(select way,height from contours WHERE height::integer % 50 = 0 AND height::integer % 100 != 0) as "contours-50"</Parameter>
                &datasource-settings;
        </Datasource>
</Layer>
<Layer name="srtm_100" status="on" srs="+proj=latlong +datum=WGS84">
        <StyleName>contours100</StyleName>
        <StyleName>contours-text100</StyleName>
        <Datasource>
                <Parameter name="table">(select way,height from contours WHERE height::integer % 100 = 0) as "contours-100"</Parameter>
                &datasource-settings;
        </Datasource>
</Layer>

Hillshading

http://wiki.openstreetmap.org/wiki/HikingBikingMaps#Hill_Shading

Serve Tiles

Tiles can simply be served by Apache:

vim /etc/apache2/sites-available/mysite
<VirtualHost *:80>
  ServerName mysite.org
  Alias /tiles     /var/www/tiles
  <LocationMatch "/tiles/">
    Order Allow,Deny
    Allow from all
  </LocationMatch>
</VirtualHost>
apache2ctl restart

WMS

In order to serve the OSM data via WMS then can use mod_mapnik_wms:

http://wiki.openstreetmap.org/wiki/Mod_mapnik_wms

gpg --gen-key
apt-get install pbuilder debsigs apache2-prefork-dev libmapnik-dev libgd2-xpm-dev
svn export http://svn.openstreetmap.org/applications/utils/mod_mapnik_wms
cd mod_mapnik_wms
sh autogen.sh
vim debian/mapnik_wms.load
LoadFile /usr/lib/libmapnik.so.0.7

debuild

This data can then be converted by GeoServer into a KML SuperOverlay to display in Google Earth:

Rails Port

This is needed to allow editing of the data using Potlatch, JOSM, etc

http://wiki.openstreetmap.org/wiki/The_Rails_Port

apt-get install imagemagick libmagick9-dev
apt-get install ruby ruby1.8-dev libxml2-dev libxml-ruby1.8 libxml-parser-ruby1.8 rubygems librmagick-ruby
gem install -v=2.3.8 rails 
#gem install libxml-ruby
#gem install composite_primary_keys
#gem install rmagick
gem install timecop
gem install pg
gem install oauth

#svn co http://railsexpress.de/svn/plugins/sql_session_store/trunk sql_session_store

apt-get install postgresql-contrib libpq-dev
su postgres
createuser openstreetmap -s -P
createdb -E UTF8 -O openstreetmap openstreetmap 
createdb -E UTF8 -O openstreetmap osm_test
createdb -E UTF8 -O openstreetmap osm
psql -d openstreetmap < /usr/share/postgresql/8.4/contrib/btree_gist.sql

apt-get install git
cd /home
git clone git://git.openstreetmap.org/rails.git
cd rails
cp config/postgres.example.database.yml config/database.yml
vim config/database.yml

rake gems:install
rake db:migrate
env RAILS_ENV=production rake db:migrate
rake test

osmosis --read-xml-0.6 file="planet.osm.bz2" --write-apidb-0.6 populateCurrentTables=yes host="localhost" database="openstreetmap" user="openstreetmap" password="openstreetmap" validateSchemaVersion=no

select setval('acls_id_seq', (select max(id) from acls)); 
select setval('changesets_id_seq', (select max(id) from changesets)); 
select setval('countries_id_seq', (select max(id) from countries)); 
select setval('current_nodes_id_seq', (select max(id) from current_nodes)); 
select setval('current_relations_id_seq', (select max(id) from current_relations)); 
select setval('current_ways_id_seq', (select max(id) from current_ways)); 
select setval('diary_comments_id_seq', (select max(id) from diary_comments)); 
select setval('diary_entries_id_seq', (select max(id) from diary_entries)); 
select setval('friends_id_seq', (select max(id) from friends)); 
select setval('gpx_file_tags_id_seq', (select max(id) from gpx_file_tags)); 
select setval('gpx_files_id_seq', (select max(id) from gpx_files)); 
select setval('messages_id_seq', (select max(id) from messages)); 
select setval('sessions_id_seq', (select max(id) from sessions)); 
select setval('user_tokens_id_seq', (select max(id) from user_tokens)); 
select setval('users_id_seq', (select max(id) from users));

cd /home/rails
ruby script/server

cd db/functions
make libpgosm.so

    * Log into PgSQL and execute the CREATE FUNCTION statement from maptile.c's comment: 

 CREATE FUNCTION maptile_for_point(int8, int8, int4) RETURNS int4 
 AS '/path/to/rails-port/db/functions/libpgosm', 'maptile_for_point'
 LANGUAGE C STRICT;

 CREATE FUNCTION tile_for_point(int4, int4) RETURNS int8 
 AS '/path/to/rails-port/db/functions/libpgosm', 'tile_for_point'
 LANGUAGE C STRICT;

Vector Overlays

Can have OSM Vectors displayed over the top of other Base Layers (e.g. Satellite Images)

BluePrint/GIS/OSMLayers

Import

We have an XSLT stylesheet to import .osm files

e.g for hospitals and clinics:

osmosis --read-xml country.osm --tf accept-nodes amenity=hospital,clinic --tf reject-ways --tf reject-relations --write-xml nodes.osm
osmosis --read-xml country.osm --tf reject-relations --tf accept-ways amenity=hospital,clinic --used-node --write-xml ways.osm
osmosis --rx nodes.osm --rx ways.osm --merge --wx country_hospitals.osm
http://myhost.com/eden/hms/hospital/create.osm?filename=country_hospitals.osm

This needs more work to understand the admin hierarchy properly to be able to import Places.

Geofabrik have updated extracts daily for Pakistan:

http://download.geofabrik.de/osm/asia/pakistan.osm.bz2

Otherwise pull a BBOX directly using Osmosis:

http://wiki.openstreetmap.org/wiki/Osmosis

Osmosis requires Java. Python options for filtering based on tag, which would be more suitable for integration within Sahana, however we need to add Polygon filtering using Shapely:

Ruby script to generate KML of recently-added locations by a group of users:

http://github.com/vertis/osm_export/tree/master

Polygons:

http://help.openstreetmap.org/questions/8273/how-do-i-extract-the-polygon-of-an-administrative-boundary

Basemap for Garmin GPS

UserGuidelinesGISDataOpenStreetMapGarmin

Mirror

How to set up a local mirror for Africa with selected Feature types, suitable for use by Import PoI:

apt-get install -y postgresql-8.4-postgis postgresql-contrib
mkdir -p /home/osm/planet/replication
cd /home/osm
wget http://dev.openstreetmap.org/~bretth/osmosis-build/osmosis-latest.tgz
tar zxvf osmosis-latest.tgz
ln -sf /home/osm/osmosis-0.41 /home/osm/osmosis
chown -R postgres /home/osm
su postgres
psql
CREATE USER osm WITH PASSWORD 'planet';
\q
createdb -O osm -E UTF8 osm -T template0
createlang plpgsql -d osm
psql -d osm -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql
psql -d osm -f /usr/share/postgresql/8.4/contrib/postgis-1.5/spatial_ref_sys.sql
psql -d osm -f /usr/share/postgresql/8.4/contrib/hstore.sql
psql -d osm -f /home/osm/osmosis/script/pgsnapshot_schema_0.6.sql
psql
\c osm
ALTER TABLE geography_columns OWNER TO osm;
ALTER TABLE geometry_columns OWNER TO osm;
ALTER TABLE nodes OWNER TO osm;
ALTER TABLE relation_members OWNER TO osm;
ALTER TABLE relations OWNER TO osm;
ALTER TABLE schema_info OWNER TO osm;
ALTER TABLE spatial_ref_sys OWNER TO osm;
ALTER TABLE users OWNER TO osm;
ALTER TABLE way_nodes OWNER TO osm;
ALTER TABLE ways OWNER TO osm;
\q
exit

cd /home/osm/planet
wget http://download.geofabrik.de/openstreetmap/africa.osm.pbf
/home/osm/osmosis/bin/osmosis --read-pbf /home/osm/planet/africa.osm.pbf --tf accept-nodes amenity=hospital,place_of_worship,school --tf accept-ways amenity=hospital,place_of_worship,school --write-pbf /home/osm/planet/eurosha.osm.pbf
su postgres
/home/osm/osmosis/bin/osmosis --read-pbf /home/osm/planet/eurosha.osm.pbf --write-pgsql database=osm user=osm password=planet

cd /home/osm/planet/replication
/home/osm/osmosis/bin/osmosis --rrii
sudo -H -u postgres psql -d osm -c "select max(tstamp) from nodes";
http://toolserver.org/~mazder/replicate-sequences/
Put timestamp into site, select hourly
vim state.txt
# paste in contents from website
vim configuration.txt
baseUrl=http://planet.openstreetmap.org/replication/hour

/home/osm/osmosis/bin/osmosis --rri workingDirectory=/home/osm/planet/replication/ --simc --read-pbf /home/osm/planet/eurosha.osm.pbf --ac --tf accept-nodes amenity=hospital,place_of_worship,school --tf accept-ways amenity=hospital,place_of_worship,school --tf reject-relations --write-pbf /home/osm/planet/eurosha-new.osm.pbf
/home/osm/osmosis/bin/osmosis --read-pbf /home/osm/planet/eurosha-new.osm.pbf --read-pbf /home/osm/planet/eurosha.osm.pbf --dc --write-pgsql-change database=osm user=osm password=planet
mv /home/osm/planet/eurosha-new.osm.pbf /home/osm/planet/eurosha.osm.pbf
vim  /home/osm/replicate.sh
#!/bin/sh
n=`ps -ef | grep -v grep | grep /home/osm/osmosis/ | wc -l`
if [ $n -le 0 ]
then
        if [ -s /home/osm/planet/eurosha-new.osm.pbf ]
        then
        mv /home/osm/planet/eurosha-new.osm.pbf /home/osm/planet/eurosha.osm.pbf
fi
/home/osm/osmosis/bin/osmosis --rri workingDirectory=/home/osm/planet/replication/ --simc --read-pbf /home/osm/planet/eurosha.osm.pbf --ac --tf accept-nodes amenity=hospital,place_of_worship,school --tf accept-ways amenity=hospital,place_of_worship,school --tf reject-relations --write-pbf /home/osm/planet/eurosha-new.osm.pbf
/home/osm/osmosis/bin/osmosis --read-pbf /home/osm/planet/eurosha-new.osm.pbf --read-pbf /home/osm/planet/eurosha.osm.pbf --dc --write-pgsql-change database=osm user=osm password=planet
mv /home/osm/planet/eurosha-new.osm.pbf /home/osm/planet/eurosha.osm.pbf
fi

chmod 0755 /home/osm/replicate.sh
vim /etc/crontab
1 *   * * * root    /home/osm/replicate.sh

Monitor:

sudo -H -u postgres psql -d osm -c "select max(tstamp) from nodes";

---

GIS

Attachments (1)

chiwogs.py (2.3 KB ) - added by Fran Boon 4 years ago. Script to export Bhutan L3s from OSM

Download all attachments as: .zip

Note: See TracWiki for help on using the wiki.

Download in other formats:

Plain Text