Schedule Loading of Shapefile into a PostgreSQL Database
The code below automatically loads the shapefile found in a certain directory into a PostgreSQL database within a given interval(in this example 2 minutes).
Downside (in which I'm trying to figure out):
What if the filename of the shapefile changes?
Anyway, this suits of what I need right now.
Change the model name and path/directory. As you noticed that if the table has an existing rows, it performed a TRUNCATE query to save memory and the system that I'm developing need the latest Flood Hazard Map. And also, it gives an error if you load a shapefile on the table with existing rows (Why?).
Next task to do is, check if the file has been modified otherwise stop the schedule.
This is based on this question and using Advanced Python Scheduler and LayerMapping utility of Django.
UPDATE: Here is the link of the updated code.
Downside (in which I'm trying to figure out):
What if the filename of the shapefile changes?
Anyway, this suits of what I need right now.
__author__ = 'Matt' import os from django.db import connection from django.contrib.gis.utils import LayerMapping from .models import FloodHazard import datetime import time import logging from apscheduler.scheduler import Scheduler logging.basicConfig() # Start the scheduler sched = Scheduler() sched.daemonic = False sched.start() #ogrinspect evidensapp/data/flood_hazard/FloodHazard.shp FloodHazard --srid=32651 --mapping --multi # Auto-generated `LayerMapping` dictionary for FloodHazard model floodhazard_mapping = { 'hazard': 'Hazard', 'date_field': 'Date_', 'geom': 'MULTIPOLYGON' } bound_shape = os.path.abspath(os.path.join(os.path.dirname(__file__), 'data/flood_hazard/FloodHazard.shp')) exist = os.path.isfile(bound_shape) if exist: def run(verbose=True): try: cursor = connection.cursor() cursor.execute("TRUNCATE TABLE evidensapp_floodhazard RESTART IDENTITY") lm = LayerMapping(FloodHazard, bound_shape, floodhazard_mapping, transform=False, encoding='iso-8859-1') lm.save(strict=True, verbose=verbose) print(datetime.datetime.now()) print(exist) time.sleep(60) except FloodHazard.DoesNotExist: lm = LayerMapping(FloodHazard, bound_shape, floodhazard_mapping, transform=False, encoding='iso-8859-1') lm.save(strict=True, verbose=verbose) print(datetime.datetime.now()) time.sleep(60) # Schedules job_function to be run every 2 minutes sched.add_interval_job(run, minutes=2) else: print(exist) sched.shutdown()
Change the model name and path/directory. As you noticed that if the table has an existing rows, it performed a TRUNCATE query to save memory and the system that I'm developing need the latest Flood Hazard Map. And also, it gives an error if you load a shapefile on the table with existing rows (Why?).
This is based on this question and using Advanced Python Scheduler and LayerMapping utility of Django.
UPDATE: Here is the link of the updated code.