Skip to main content

Updating OSM data in PostgreSQL snapshot schema from latest pbf

Getting pbf files from Geofabrik and uploading them into your own datastore has been discussed in this blog earlier. In this blog, we are gonna see how to update the data, as OSM data changes every day by millions of controbutors worldwide. 

It is important to preseve the most recent pbf file in your hard drive, as we'll see in further.

The steps to be followed are the following,

  1. Download the latest pbf file as per your business requirement.
  2. Generate the changes between the last pbf and the earlier version.
  3. Apply the change to your PostgreSQL data store.

Let's follow this with a concrete example, 
  1. Assuming the last pbf we have applied is, india-20220122.osm.pbf dated Jan, 22, 2022.
  2. Download the most recent india osm pbf file, india-latest.osm.pbf
  3. Generate the change-log in xml format,
    4. osmosis-0.48.0/bin/osmosis --read-pbf file="india-latest.osm.pbf" --read-pbf file="india-20220122.osm.pbf" --derive-change --write-xml-change file=inddiff.osc
  4. Aplpy the changes in your local postgreSQL database,
    5. osmosis-0.48.0/bin/osmosis --read-xml-change file="inddiff.osc" --write-pgsql-change database=pgsnapshot host=localhost user=postgres password=postgres
With these steps completed, you should have updated data in your local database. 
     
Labels:

Comments

Post a Comment

Popular posts from this blog

Multimaster replication with Symmetric DS

Symmetric DS is an awesome tool for trigger based replication whcih works for all major database vendors, including but not limited to PostgreSQL, MySQL, MSSQL, Oracle and many others. Symmetric-DS is a java application and can execute on any platform on whcih JRE is available including Windows and Linux. Trigger based replication, in constrast to disk based (eg. DRBD ) or transaction log file shipping based or statement based , works by registering triggers on DMLs and sending the data thus generated to remote machines. Another very popular trigger based DB replication tool is Slony . Symmetric-DS in addition to being database agnostic also supports multi-master replication (MMR). MMR usecase involves multiple database nodes, connected in a pool with DML updates coming from any of them. This is different from the normal master-slave replication, where slaves are not expected to generate any data events, and the sole authority of database is the master. MMR requirement causes d...
1 comment
Read more

Reset root password RHEL/Rocky/CentOS 9

Unlike the earlier versions of Rethat variants, version 9 doesn't allow single user mode to change password, as maintanance mode in 9 requires root password . Single user mode (runlevel 1) can easily be obtained by appending the word ' single ' at the end of the line starting with 'linux' by editing the entry in boot menu by pressing ' e ' at boot menu. To reset the root password on the other hand, one requires to follow a specific set of commands, At the boot menu, edit rescue mode to append 'rd.break ' at the end of the line starting with kernel. Boot with the edited line by pressing Ctrl+X or F10. At the new prompt starting with switch_root, type the following commands, mount -o remount, rw /sysroot chroot /sysroot touch /.autorelabel passwd <new root password> exit reboot       
Post a Comment
Read more

Devstack installation with cells in multiple machines

  Devstack is a testing and development package for OpenStack . The official devstack website has excellent but terse installation instructions, which could be daunting to a newbie. Normally, a traditional install is pretty straightforward but problems creep in when you have to go for some particular configuration or installation like cells or a simple code-test-debug setup, the information on the main site is insufficient. In this blog, I will enumerate the steps for setting up a cell-ular devstack installation with one parent and one child. Parent will not be instantiating any VMs s it's only a 'command centre'. 1. On both machines, install git and vim, clone the devstack repo using git. yum install git vim git clone https://git.openstack.org/openstack-dev/devstack 2. On both machines, open up the file 'local.conf' inside the cloned directory 'devstack' cd devstack; vim local.conf 3. Copy the parameters below to the file...
1 comment
Read more