Posts

Showing posts with the label PostgreSQL

Optimizing Near-real Time Module

Image
Super-Related Post:  GEODJANGO + GEOSERVER + OPENLAYERS COMBO! The scenario is we have a python script that runs on a given interval that automatically loads shapefile into PostgreSQL with PostGIS database.This shapefile is from a simulation done by a computer and it is updated on a timely basis. Near-real Time module of the app performs INTERSECT query to generate statistics. This involves two tables on the database. Every query took more than 5 minutes which is unacceptable. No matter how I try to change(optimize) my query, it's still SLOOOOOOOOOOOOOOOOOOOOOOW! So, instead of directly performing INTERSECT query between two tables every time the user take request. I have this instead: I created a stats table where the results are inserted and stored on it. After loading the shapefile into the database, a PostgreSQl function will be called and perform the INTERSECT query then insert the new result to the stats table. The database server will just perform select quer...

GeoDjango + GeoServer + Openlayers Combo!

Image
I am developing this web mapping application using Django(GeoDjango), Openlayers and PostgreSQL(PostGIS plugin). It is quite good but when having a huge set of data(shapefile/rows in the table/polygons with multiple vertices) it is "laggy". Panning and browsing on the map and rendering of layers is too slow. I have read some articles about it that although Openlayers has no limitation in rendering a vector(feature) layer, the browser does. So, I decided to use GeoServer. Here's my framework for now: Before, I get my GeoJSON data from Django and rendered it directly using Openlayers but since it is, as I said "laggy" the visualization of the map now is from GeoServer (WMS). HUGE difference, in terms of its rendering performance. It is quite good but I need to learn more about tuning it and use native JAI for rendering images. And also, performing PostGIS queries(ST_Intersects, etc...) is slow, I have tried using raw sql for it but still the response...

Schedule Loading of Shapefile into a PostgreSQL Database

The code below automatically loads the shapefile found in a certain directory into a PostgreSQL database within a given interval(in this example 2 minutes). Downside (in which I'm trying to figure out): What if the filename of the shapefile changes? Anyway, this suits of what I need right now. __author__ = 'Matt' import os from django.db import connection from django.contrib.gis.utils import LayerMapping from .models import FloodHazard import datetime import time import logging from apscheduler.scheduler import Scheduler logging.basicConfig() # Start the scheduler sched = Scheduler() sched.daemonic = False sched.start() #ogrinspect evidensapp/data/flood_hazard/FloodHazard.shp FloodHazard --srid=32651 --mapping --multi # Auto-generated `LayerMapping` dictionary for FloodHazard model floodhazard_mapping = { 'hazard': 'Hazard', 'date_field': 'Date_', 'geom': 'MULTIPOLYGON' } bound_shape = os.path.abspath(os....