Skip to main content

Opensteetmap Data Models

Opensteetmap is the crowdsourced mapping of the planet and can be accessed at https://opensteetmap.org

The map data is freely downloadable and can be hosted locally for any purpose whatsoever under GPL license. Map data is downloadable in multiple archive formats like pbf (Protocolbuffer Binary Format) and others.


OSM's internal data model is represented as shown in the ER diagram above, but when it's imported from the PBF, it has to be efficient, fast and easy to use. Primarily, OSM databases can either be,

  1. Snapshot or 
  2. Historical
Snapshot databases don't contain historical infromation (for instance, how a river changed it's course of the years or how landuse of a forested land changed over the year) whereas, Historical database contains this information. Naturally, historical databases are much more complex and requires more effort.

Various parameters on how to judge various data stores are,
  1. Updatability
  2. Transactibility
  3. Concurrent Read/Writability
  4. Snapshot/Historical ?
Few of the various data store usercases used by OSM community and developers are,
  1. API DB - This is the original PostgreSQL (not PostGIS) database running at the main OSM server.
    1. Updatable
    2. Full transaction support
    3. Full concurrent reads/writes
    4. Full historical data
  2. osm2pgsql - This is the primary datastore used by renderers like Mapnik.
    1. Updatable
    2. No transaction support
    3. Single writer/multiple reader
    4. Snapshot
  3. pgsnapshot - This is a popular schema generated by osmosis tool. It's mainly used for analysis.
    1. Updatable
    2. Snapshot
  4. Normatim -  This is a schema used primarily for geocoding and searching. Generated by osm2pgsql
    1. Updatable
    2. Snapshot
It's important to choose datastore that is appropriate to the usecase, because GIS queries can be damn expensive especially when the records can be in multiple gigabytes.

Labels:

Comments

Post a Comment

Popular posts from this blog

Multimaster replication with Symmetric DS

Symmetric DS is an awesome tool for trigger based replication whcih works for all major database vendors, including but not limited to PostgreSQL, MySQL, MSSQL, Oracle and many others. Symmetric-DS is a java application and can execute on any platform on whcih JRE is available including Windows and Linux. Trigger based replication, in constrast to disk based (eg. DRBD ) or transaction log file shipping based or statement based , works by registering triggers on DMLs and sending the data thus generated to remote machines. Another very popular trigger based DB replication tool is Slony . Symmetric-DS in addition to being database agnostic also supports multi-master replication (MMR). MMR usecase involves multiple database nodes, connected in a pool with DML updates coming from any of them. This is different from the normal master-slave replication, where slaves are not expected to generate any data events, and the sole authority of database is the master. MMR requirement causes d...
1 comment
Read more

RabbitMQ and SSL

RabbitMQ is an AMQP provider i.e. it can reliably queue, service and maintain messages according to a range of policies and parameters. By default, it listens to plain old TCP connections and sends and receives messages over plaintext. This feature just works "out of the box". For users who wish to use SSL over TCP aka TLS, it requires a bit more work on their part. First, let's create a bunch of certificates and sign them with our own CA. For this, we'll use easyrsa3 . Easyrsa is a CLI tool to create, sign and manage your own certification authorities. It's maintained by OpenVPN team. Download easyrsa using your native package manager i.e. yum or apt-get $cp -Rp /usr/share/easy-rsa ~/easy-rsa-3   $cd ~/easy-rsa-3 $./easyrsa init-pki $./easyrsa build-ca $./easyrsa build-server-full broker [nopass] $./easyrsa build-client-full client1 [nopass] This creates three entities (collection of private keys, public keys and certificates) for a CA, a s...
Post a Comment
Read more

Motorola XT502 Custom ROMs

I purchased an Android Phone in the early days of it coming to Indian market and I able to afford it ( just got a job), during mid 2010. The recent and popular version was Éclairs. I just went up to the shop and bought a nice and shiny Motorola Quench XT3 (XT502). When in other places XT502 was having Andriod Donut, I got as a special offer Éclairs. I was happy as hell. The Droid A couple of year passes, and newer versions of Android came from Éclairs to Froyo to Gingerbread then to the bigger version upgrades like Honeycomb, Ice Cream Sandwich and then Jelly Bean. By the end of 2012, I was literally surviving on my Motorola with Éclairs. I had to upgrade, anyhow. Now more that Motorola denied any upgrades for Quench XT3 . A trivia, Android version names are taken from sugary desserts with lexicographic sequencing. So, I did up-gradation from Éclairs to Gingerbread using a custom ROM from Cyanogenmod . An excellent community of enthusiast who develop their own custom ROMs oft...
5 comments
Read more