OSGeo Planet

GeoServer Team: GeoServer 2.13.1 Released

OSGeo Planet - Mon, 2018-05-21 20:07

We are happy to announce the release of GeoServer 2.13.1. Downloads are available (zipwar, and exe) along with docs and extensions.

This is a stable release recommended for production use. This release is made in conjunction with GeoTools 19.1 and GeoWebCache 1.13.1.

Highlights of this release are featured below, for more information please see the release notes (2.13.1 | | 2.13-beta):

New Features and Improvements
  • MongoDB community module moved to extension
  • Support PNG/JPEG WPS Downloads
  • Allow self joining GetFeature without aliases
  • Add support for priority in control-flow bounded queues
  • Hibernate Monitoring extension moved to a community module
Bug Fixes
  • WCS 1.0.0 generates wrong links in GetCapabilities
  • WFS 2.0 capabilities report transaction support even if the service level is not configured as such
  • WPSResourceManager cleanup is not deleting temporary subfolders (only files)
  • GeoServer in CITE compliance mode fails to validate an empty LockFeature request
  • WMS 1.3 GetMap request significantly slower than WMS 1.1.0 since GeoServer 2.11.4
  • Import objects cannot be deleted when in COMPLETE state
  • Style with Interpolate function causes NullPointerException on GetLegendGraphic
  • Windows Installation issue – upgrading GeoServer results in corrupt data_dir
  • Windows Installer: Remove ‘work’ folder when uninstalling


Categories: OSGeo Planet

GeoTools Team: GeoTools 19.1 Released

OSGeo Planet - Mon, 2018-05-21 19:56
The GeoTools team is pleased to announce the release of GeoTools 19.1: geotools-19.1-bin.zip geotools-19.1-doc.zip geotools-19.1-userguide.zip geotools-19.1-project.zip maven repository This release is the current stable release and as such users and downstream projects should consider moving from older releases to this one. This release is made in conjunction with GeoServer 2.13 and
Categories: OSGeo Planet

gvSIG Team: Participación en el 3er Congreso Nacional de Estudiantes de Geomática, Geodesia y Geoinformática de México

OSGeo Planet - Mon, 2018-05-21 16:18

Ya de regreso del congreso que reúne a estudiantes de geomática, geodesia y geoinformática de todo México. no podemos estar más satisfechos con los resultados. Durante el mismo no sólo pudimos comprobar el interés creciente por gvSIG en la comunidad universitaria e introducir a los estudiantes en su uso por medio de un taller que se quedó sin plazas libres, sino también algo tan importante como intercambiar impresiones con estudiantes, profesores y egresados que ahora en sus puestos de trabajo utilizan la suite gvSIG.

Otro hito a destacar es el acuerdo de colaboración alcanzado entre la Asociación gvSIG y la UAEMEX y que conlleva entre otras actividades la realización de prácticas de los alumnos en la Asociación gvSIG, así como la formación certificada del profesorado. Un convenio similar al firmado también con la UAS. Esperamos que este tipo de convenios se repliquen en universidades de todo el mundo.

Sumando todos somos más fuertes.

Categories: OSGeo Planet

QGIS Blog: New PyQGIS documentation

OSGeo Planet - Sun, 2018-05-20 17:42

We are proud to announce our new dedicated documentation of the QGIS Python API (also called PyQGIS) which is now available at https://qgis.org/pyqgis:

While the QGIS API has long been documented, Python developers in the past had to work with the general C++ documentation that wasn’t always straightforward to use. The new PyQGIS documentation presents the API in an accessible pythonic manner.

Of course, creating a good API documentation from source code in an automated way, is not trivial. A key challenge was to automatically create Python bindings files (or SIP files). A custom Perl script known as “sipify” now enables us to automatically integrate the C++ documentation into the Python bindings and keep them up to date. Another challenge was to create the documentation itself using Sphinx. Two detailed reports containing all the technical details of the first and second generation of the documentation are available if you want to learn more about the underlying architecture.

This has been a really important infrastructure project for QGIS that has been made possible by support from our donors and sponsors, as well as the generous in-kind contributions of our community members.

Categories: OSGeo Planet

GeoSolutions: GeoSolutions is sponsoring FOSS4G Europe 2018!

OSGeo Planet - Sun, 2018-05-20 13:57

FOSS4G EU 2018

Dear Reader,

GeoSolutions is proud to announce that we will take part in this year FOSS4G Europe in Guimaraes, Portugal, from 16th to 21nd July as Silver Sponsor.

We will also give workshops and presentations covering MapStore, GeoServer and GeoNode; we will provide more details once the the full program has been announced. If you want further information, do not hesitate to contact us.

Looking forward to seeing you in Guimaraes!

The GeoSolutions Team,

Categories: OSGeo Planet

Free and Open Source GIS Ramblings: Scalable spatial vector data processing

OSGeo Planet - Fri, 2018-05-18 20:07

Working with movement data analysis, I’ve banged my head against performance issues every once in a while. For example, PostgreSQL – and therefore PostGIS – run queries in a single thread of execution. This is now changing, with more and more functionality being parallelized. PostgreSQL version 9.6 (released on 2016-09-29) included important steps towards parallelization, including parallel execution of sequential scans, joins and aggregates. Still, there is no parallel processing in PostGIS so far (but it is under development as described by Paul Ramsey in his posts “Parallel PostGIS II” and “PostGIS Scaling” from late 2017).

At the FOSS4G2016 in Bonn, I had the pleasure to chat with Shoaib Burq who ran the “An intro to Apache PySpark for Big Data GeoAnalysis” workshop. Back home, I downloaded the workshop material and gave it a try but since I wanted a scalable system for storing, analyzing, and visualizing spatial data, it didn’t really seem to fit the bill.

Around one year ago, my search grew more serious since we needed a solution that would support our research group’s new projects where we expected to work with billions of location records (timestamped points and associated attributes). I was happy to find that the fine folks at LocationTech have some very promising open source projects focusing on big spatial data, most notably GeoMesa and GeoWave. Both tools take care of storing and querying big spatio-temporal datasets and integrate into GeoServer for publication and visualization. (A good – if already slightly outdated – comparison of the two has been published by Azavea.)

My understanding at the time was that GeoMesa had a stronger vector data focus while GeoWave was more focused on raster data. This lead me to try out GeoMesa. I published my first steps in “Getting started with GeoMesa using Geodocker” but things only really started to take off once I joined the developer chats and was pointed towards CCRI’s cloud-local “a collection of bash scripts to set up a single-node cloud on your desktop, laptop, or NUC”. This enabled me to skip most of the setup pains and go straight to testing GeoMesa’s functionality.

The learning curve is rather significant: numerous big data stack components (including HDFS, Accumulo, and GeoMesa), a most likely new language (Scala), as well as the Spark computing system require some getting used to. One thing that softened the blow is the fact that writing queries in SparkSQL + GeoMesa is pretty close to writing PostGIS queries. It’s also rather impressive to browse hundreds of millions of points by connecting QGIS TimeManager to a GeoServer WMS-T with GeoMesa backend.

Spatial big data stack with GeoMesa

One of the first big datasets I’ve tested are taxi floating car data (FCD). At one million records per day, the three years in the following example amount to a total of around one billion timestamped points. A query for travel times between arbitrary start and destination locations took a couple of seconds:

Travel time statistics with GeoMesa (left) compared to Google Maps predictions (right)

Besides travel time predictions, I’m also looking into the potential for predicting future movement. After all, it seems not unreasonable to assume that an object would move in a similar fashion as other similar objects did in the past.

Early results of a proof of concept for GeoMesa based movement prediction

Big spatial data – both vector and raster – are an exciting challenge bringing new tools and approaches to our ever expanding spatial toolset. Development of components in open source big data stacks is rapid – not unlike the development speed of QGIS. This can make it challenging to keep up but it also holds promises for continuous improvements and quick turn-around times.

If you are using GeoMesa to work with spatio-temporal data, I’d love to hear about your experiences.

Categories: OSGeo Planet

From GIS to Remote Sensing: Download products: issue with Landsat previews

OSGeo Planet - Fri, 2018-05-18 18:35
This post is to inform you about an error preventing the Landsat previews to be displayed in the Download tab.
After the search, clicking any preview will cause the error "Unable to connect".

The error is caused by changes of the Landsat Collection 1 preview url in the site earthexplorer.usgs.gov . The search queries to Earthdata earthdata.nasa.gov still return the old urls, causing this issue.
I hope that they will fix this soon, but at the moment the Landsat previews aren't available in SCP, therefore the only solution to download Landsat images is to disable the option "only if preview in Layers" . This way you should be able to download all the images listed in the table.

For any comment or question, join the Facebook group and the Google+ Community about the Semi-Automatic Classification Plugin.
Categories: OSGeo Planet

CARTO Inside Blog: CARTO in QGIS using OGR

OSGeo Planet - Thu, 2018-05-17 10:39

You can use OGR to move data into and out of CARTO. And you can use QGIS to view and edit layers supported by OGR. So it would stand to reason that you should be able to use QGIS to view and edit CARTO data directly: but how?

Here’s one quick and dirty way to connect QGIS to your CARTO layers.


First, you need to make sure QGIS can access your layers using a CARTO master API key. The OGR driver reads system tables, so it requires the master key to operate.

QGIS Environment Variables

  • Open the QGIS Preferences menu.
  • Navigate to the System panel.
  • Scroll to the Environment area.
  • Add a new environment variable, CARTO_API_KEY.
  • Put your API key in the “Value” field.
  • Press the OK button.
  • Shut down QGIS and re-open it to bring the new environment variable into effect.

Now we need to record the connection information OGR will need to access CARTO, and put that information into a “VRT” file.

VRT File

A VRT file defines connection information and layer names so that an OGR client (like QGIS) can easily connect to a source without reading a lot of metadata. Here’s an example minimal VRT file with two layers defined:

<OGRVRTDataSource> <OGRVRTLayer name="subway_stations"> <LayerSRS>EPSG:4326</LayerSRS> <SrcDataSource>Carto:pramsey</SrcDataSource> <SrcLayer>nyc_subway_stations</SrcLayer> <GeometryType>wkbPoint</GeometryType> </OGRVRTLayer> <OGRVRTLayer name="streets"> <LayerSRS>EPSG:4326</LayerSRS> <SrcDataSource>Carto:pramsey</SrcDataSource> <SrcLayer>nyc_streets</SrcLayer> <GeometryType>wkbLineString</GeometryType> </OGRVRTLayer> </OGRVRTDataSource>
  • This file reads CARTO tables nyc_subway_stations and nyc_streets and exposes them to QGIS using the names “subway_stations” and “streets”.
  • The <LayerSRS> should always be EPSG:4326, as that is the system CARTO always uses.
  • The <SrcDataSource> is of the form “Carto:username”, where “Carto” tells OGR what driver to use and the “username” is your CARTO user name. For multi-user accounts, “username” must be the user name and not the organization name.
  • The <GeometryType> is optional, but ensures that OGR knows whether the input layer is a point, line or polygon.

You can test your VRT file using the ogrinfo utility. You should be able to run ogrinfo and get a listing of layers back, for example:

# ogrinfo carto.vrt INFO: Open of `carto.vrt' using driver `OGR_VRT' successful. 1: subway_stations 2: streets

Once you have a working VRT file, you can add the file as a layer in QGIS!

Adding the Layer

After you’ve set up your API key and authored your VRT file, go to Layer > Add Layer > Add Vector Layer… in QGIS, and select your VRT as the source vector dataset, using a “File” source type.

QGIS Vector File Source

If your VRT includes multiple layer definitions, you’ll be asked to select which layers (or all of them) that you want to add. Then you should be able to see the data draw on your QGIS map!

QGIS Vector File Source

Working with the Layers

Once you have loaded the layers, they work just like any other QGIS layer:

  • You can style them any way you like.
  • You can include them in printed output.
  • You can reproject the map and see them in other projections.
  • You can edit them (yes, really!)
  • You can include them in QGIS analyses.

Since you are editing the live data in CARTO, it’s possible to apply edits in QGIS and see your published CARTO maps update in real time!

Categories: OSGeo Planet

gvSIG Team: Encuesta sobre gvSIG Online

OSGeo Planet - Wed, 2018-05-16 10:50

Con el objetivo de mejorar gvSIG Online, os pedimos que dediquéis unos pocos minutos a completar una pequeña encuesta. Todas las respuestas serán tratadas de forma confidencial y no serán utilizadas para ningún propósito distinto a la investigación que estamos realizando sobre gvSIG Online. La encuesta consta de 20 preguntas y te dedicará unos 5 minutos el completarla.

Para acceder a ella pulse en el siguiente enlace: https://devel.gvsigonline.com/limesurvey/index.php/974958?newtest=Y&lang=es

El plazo límite para completar la encuesta es el 25 de mayo. ¡Gracias por colaborar!

Categories: OSGeo Planet

gvSIG Team: Certification of the free course about crime mapping. GIS for criminology

OSGeo Planet - Wed, 2018-05-16 02:23

After the good reception of the course about GIS applied to criminology, many of you have asked us about a course completion certificate.

We have prepared an exercise that you have to complete and send it to us to be evaluated in order to get the certificate. The cost of the certification has been reduced considerably with the objective to allow anyone to get it.

All the information about the exercise, the mail address to send it, and the payment is available at the PDF file at this link.

Categories: OSGeo Planet

Petr Pridal: Streets: new OpenMapTiles map style

OSGeo Planet - Tue, 2018-05-15 11:00

OpenMapTiles has always encouraged the development of map services by giving an option to build a self-hosted map with open-source tools and by providing free non-commercial hosting. For those who support us with their paid plan, we introduce another premium service: a new Street map style.

Map with a clear roads’ hierarchy

Each map style should highlight only the information relevant to its primary objective. When we were designing new map style, we focused on transportation. Therefore the name Streets.

Streets style is a base map highlighting different transportation methods. Therefore we include roads, railway network, ferry routes, and airports. Moreover, it also shows all relevant POIs like bus stops, petrol stations or subway stations. 

Street style is a fresh alternative to Google Maps default style.

To highlight the roads’ hierarchy, each type of road is represented by different color. While the most saturated colors are representing major roads like highways, more pale colors are used in combination with thin lines for minor roads. By looking at the map, you can at the first look easily judge the overall network and make a quick decision.

Different types of transportation in Hong Kong and bilingual labels Different types of transportation in Hong Kong and bilingual labels

Only relevant information for quick decision

When looking at a map, you want to capture all necessary information as fast as possible. On the other hand, placing too many details decreases the readability and it takes you longer to get oriented on a map. Therefore we carefully picked only relevant information for each zoom level like POIs, labels, minor roads, and others. As you zoom in, more details appear they disappear as you zoom out to give you clear overlook. Since this is our first version, there could still be some missing or redundant items, especially POIs. Any comments on this topic are very welcomed.

A good map should also help the person who is using it recognize the features of the real world on the map. On lower zoom levels, we are displaying the land use information which can be seen from a global satellite image. On the higher zoom levels, we show transparent 3D buildings to help a person with orienting in an urban jungle. The buildings can be turned off.

The map is fully customizable as our MapTiler TileHosting supports creating own styles or derivatives of the existing ones with built-in WYSIWYG design tool.

As each of our map, the Streets style also includes a possibility to change the language. Currently, there are more than 50 languages supported.

Different types of transportation in Hong Kong and bilingual labels Land use information which can be seen from a satellite image from a global satellite image.

All the map styles including Streets are visible as a browsable map on the MapTiler TileHosting main page or in the administration.

Categories: OSGeo Planet

gvSIG Team: Certificación curso gratuito de Mapeo del delito. SIG en criminología

OSGeo Planet - Tue, 2018-05-15 05:05

Tras la buena recepción del curso de SIG aplicado a la criminología muchos de vosotros nos habéis preguntado por la posibilidad de obtener un certificado de aprovechamiento del curso.

Hemos preparado un ejercicio que debéis resolver y enviarnos para su evaluación y poder así obtener dicho certificado. El coste de la certificación lo hemos reducido al máximo, con el objetivo de que cualquier interesado en obtenerlo pueda acceder a él.

Toda la información sobre el ejercicio, la dirección del envío y el pago la podéis encontrar en el PDF accesible desde este enlace.

Categories: OSGeo Planet

Markus Neteler: PDAL 1.7 packaged for Fedora including vertical datums and grids

OSGeo Planet - Mon, 2018-05-14 20:47

 openNRW Germany)In order to simplify the installation of the latest PDAL release (Point Data Abstraction Library, https://pdal.io/, version 1.7.0 1.7.2) on Fedora, I have created an updated set of RPM packages, again including the vertical datums and grids available from OSGeo (i.e., .gtx files from here).

The installation is as simple as this (the repository is located at Fedora’s COPR):

# enable extra repos to satisfy dependencies sudo dnf copr enable neteler/pdal-hexer sudo dnf copr enable neteler/points2grid sudo dnf copr enable neteler/laszip # install minimal dependencies sudo dnf install hexer sudo dnf install points2grid # enable and install PDAL sudo dnf copr enable neteler/pdal sudo dnf install PDAL PDAL-vdatums # run it pdal-config --version pdal --help


The post PDAL 1.7 packaged for Fedora including vertical datums and grids appeared first on GFOSS Blog | GRASS GIS and OSGeo News.

Categories: OSGeo Planet

Paul Ramsey: PostGIS Talks @ FOSS4G North America

OSGeo Planet - Mon, 2018-05-14 16:00

I presented my “PostGIS for Managers” talk for the last time (at least in this form) today at FOSS4G North America. The reason it’s for the last time is that the central conceit it’s built around, that a core decision is between a proprietary and an open source database, isn’t really operative anymore. The real decisions are now being driven by other considerations, like cloud platforms, and the services available there. So, it’s not really PostgreSQL versus Oracle anymore.

I also presented my “SQL Festival” talk, for the first time! New material is always a little challenging: will it work, is it the right level for the audience? It seemed to be well received, a mix of generic SQL tidbits, and some specific interesting queries you can do with PostGIS.

Categories: OSGeo Planet

Andrea Antonello: Geopaparazzi 5.6.0 is out - work with cloud profiles to handle surveys and data!

OSGeo Planet - Mon, 2018-05-14 13:13
This release brings a ton of fixes and compatibility enhancements for newer Android versions.

And then it brings cloud profiles. And I think this one is huge! :-)

Cloud ProfilesIn the last months a new company entered the geopaparazzi community and they have been doing nice contributions. The guys from Geoanalytic were working with us on the implementation of a more structured version of the cloud projects synchronization: Cloud Profiles!

We have been working on this already at the Bonn code sprint together with Cesar from the company Scolab. And now we are finally at a first release that supports this concept.

Cloud Profiles are a great way of easing Geopaparazzi’s data handling tasks. When a web server is configured to serve Cloud Profiles, Geopaparazzi can automatically download Projects, Basemaps, Spatialite Overlays, forms for Notes, and other files. When a user activates a downloaded Profile, Basemaps are made available, Overlays are attached to the Map View and layers are set to display.

Read more about it here.

The Geoanalytic guys have also been so nice to write a reference geopaparazzi profile server that can be used as a starting point. You can read more about it and download it from here.

The real power behind cloud profiles is that you can use your own server for this, but you could also make it much simpler by using Generic Cloud Server. You can set up your own Cloud Profile server by putting your files on a generic cloud file server like Dropbox or Google Drive, and editing a Cloud Profiles list like the one above and putting it on the cloud file server as well.

The very important fixThe most important fix (in my opinion) is the one resolving the GPS Location Limitations on Android Oreo. Basically it wasn't possible to log gps tracks with the screen off.

Technically it has been necessary to create a notification icon for geopaparazzi, so you will now have that visible when geopaparazzi is active.

The positive side is that:
  1. you will always know when geopaparazzi is active
  2. you can always see information about your position directly in the notification area

Other features and fixesValues in settings. The settings screen now shows the actual values:

Buttons size. Small buttons are hard to see and press while on the trail. While it can be difficult to show a lot of information on a small screen, where there is room, you can now change the button and text size. This applies to the notes view for now:

Dynamic hints. Dashboard button hints are dynamic where possible. You can now see from there how many notes and logs you have in store. Just long-tap on the buttons:

Notes settings. The notes settings view is now accessible from the notes list (it was hidden in the gps data list menu). Access it from the palette icon:

PDF export. The PDF export now allows to export a subset of notes instead of everything contained in the project. The user can select the notes he/she likes to export and those will be converted in the pdf version.
Linked resources. It is now possible to view not only images stored in a Spatialite database when they are related to (geospatial) features but also for example PDF. 

Note that while a user can take pictures in the field and link them to a feature, in the case of PDF, the resource has to be linked before, i.e. it can be just viewed only from geopaparazzi.
Remove all. In the basemaps view it is now possible to remove all maps in one tap. This is really helpfull for those that are used to load maps through the load-folder option and then need a lot of time to remove them to have things more readable.

Mapurl service. One sad note is that the Tanto Mapurl service, which was used to download automagically configured mapurls based on WMS services, is no longer maintained and has therefore been removed also from geopaparazzi.

Fixes for:
  • images taken within Geopaparazzi are not geotagged.
  • unable to open form image thumbnails. 
  • many other fixes have been done, the complete list is here.

Categories: OSGeo Planet

CARTO Inside Blog: A SQL approach to graph coloring applied to maps

OSGeo Planet - Mon, 2018-05-14 11:00

At CARTO we challenge ourselves to use our platform as our users do. For us, this serves several purposes:

  • Because we care a lot about our users experience, this way we understand better the pains and gains of using our platform.
  • We still keep learning a lot: from SQL to React, going through WebGL to projections, spatial algorithms and mapping in general.

Since use cases at CARTO go from very simple visualizations to complex geospatial solutions, one of the challenges I set for myself was to solve the graph coloring problem with CARTO.

Graph coloring is a technique to assign colors to the vertices of a graph such that no two adjacent vertices share the same color.

But what does graph coloring have to do with maps?

Map coloring is an application of graph coloring so each two adjacent polygons (countries, provinces, etc.) are assigned different colors.

Map coloring helps to better understand maps and solve other kind of problems like mobile radio frequency assignment or other scheduling problems.

Having said that, graph coloring is a very interesting topic covered by several theorems and algorithms, like the 4-color theorem which basically states that any map can be colored using 4 colors.

Coloring the world map in 4 colors

Let’s put ourselves in the boots of a CARTO user that wants to draw the world map in 4 colors. We don’t have much programming skills, but we know a bit of SQL and spatial concepts.

For this case we’ll work with the world_borders layer that can be imported into any CARTO account from our Data Library.

Modelling a graph in PostGIS

The graph coloring problem needs two mathematical artifacts to be solved: a model and an algorithm.

So we have to model a PostGIS table into a graph. This may sound like complex stuff but it can be actually solved in less than 10 lines of SQL by creating an adjacency list:

SELECT DISTINCT a.name, array_agg(b.name) over (PARTITION BY a.cartodb_id) AS adjacent, count(b.*) over (PARTITION BY a.cartodb_id) AS valence, 0 AS color FROM world_borders a, world_borders b WHERE ST_INTERSECTS(a.the_geom, b.the_geom) AND a.cartodb_id != b.cartodb_id ORDER BY valence DESC, a.name ASC

With this query we obtain the world map adjacency list having for each country, the list of adjacent countries and its valence (the number of adjacent countries).

Learning point: Note the use of PostgreSQL Window functions to aggregate and count the adjacent countries in a single column. Window functions are a really handy resource to have in your SQL tool box.

We can generalize this query by wrapping it as a PostgreSQL function so that it can be executed for any of our datasets and store the results in a table:

CREATE OR REPLACE FUNCTION adjacency_list(table_name regclass, user_name text) RETURNS void AS $$ BEGIN EXECUTE format('DROP TABLE IF EXISTS %s_adjacency_list; CREATE TABLE %s_adjacency_list AS SELECT DISTINCT a.cartodb_id, array_agg(b.cartodb_id) over (PARTITION BY a.cartodb_id) AS adjacent, count(b.*) over (PARTITION BY a.cartodb_id) AS valence, 0 AS color FROM %s a, %s b WHERE ST_INTERSECTS(a.the_geom, b.the_geom) ORDER BY valence DESC, a.cartodb_id ASC; SELECT CDB_CartoDBFyTable(''' || user_name || ''',''' || table_name || '_adjacency_list'');', table_name, table_name, table_name, table_name); END $$ LANGUAGE plpgsql;

Learning point: By wrapping a query into a function we are able to re-use it and even provide of a geospatial framework to the users in our CARTO organization.

Note as well the use of the EXECUTE and format functions to avoid missusing of the function or SQL injection issues.

The Welsh-Powell algorithm: a greedy coloring approach

Let’s start by implementing the most simpler algorithm for graph coloring, the Welsh-Powell one. This algorithm is as follows:

  • Find the adjacency list and valence for each vertex (in this case for each country)
  • List the vertices in order of descending valence
  • Color the first vertex in the list with color 1
  • Go down the list and color every vertex not connected to the colored vertices above the same color. Then cross out all colored vertices in the list.
  • Repeat on the uncolored vertices with a new color, always working in descending order of valence until all the vertices have been colored.

Let’s see how we can implement the Welsh-Powell algorithm as a PostGIS function.

CREATE OR REPLACE FUNCTION greedy(table_name regclass, user_name text) RETURNS void AS $$ DECLARE item RECORD; current_color int = 1; cc int; guard int; BEGIN PERFORM adjacency_list(table_name, user_name); EXECUTE format('SELECT COUNT(*) FROM %s_adjacency_list WHERE color = 0', table_name) INTO guard; WHILE guard > 0 LOOP EXECUTE format('UPDATE %s_adjacency_list SET color = $1 WHERE cartodb_id = (SELECT cartodb_id FROM %s_adjacency_list WHERE color = 0 ORDER BY cartodb_id ASC LIMIT 1);', table_name, table_name) USING current_color; FOR item IN EXECUTE format('SELECT * FROM %s_adjacency_list WHERE color = 0', table_name) LOOP EXECUTE format('SELECT COUNT(*) FROM %s_adjacency_list WHERE color = $1 AND $2 = ANY(adjacent)', table_name) INTO cc USING current_color, item.cartodb_id; IF cc = 0 THEN EXECUTE format('UPDATE %s_adjacency_list SET color = $1 WHERE cartodb_id = $2;', table_name) USING current_color, item.cartodb_id; END IF; END loop; current_color = current_color + 1; EXECUTE format('SELECT COUNT(*) FROM %s_adjacency_list WHERE color = 0', table_name) into guard; END loop; END; $$ LANGUAGE plpgsql; A deep view on the Welsh-Powell algorithm implementation

In this case we have two different sections in our function. First we DECLARE temporary variables needed to store results and second we have the actual algorithm implementation between a BEGIN and END clause.

DECLARE item RECORD; current_color int = 1; cc int; guard int;

Since we need to model our dataset as an adjacency list we start by calling our adjacency_list function:

PERFORM adjacency_list(table_name, user_name);

Then we need to know the number of rows in the dataset and start an iterative algorithm:

EXECUTE format('SELECT COUNT(*) FROM %s_adjacency_list WHERE color = 0', table_name) INTO guard; WHILE guard > 0 LOOP

Let’s color the first vertex in the list with color 1

EXECUTE format('UPDATE %s_adjacency_list SET color = $1 WHERE cartodb_id = (SELECT cartodb_id FROM %s_adjacency_list WHERE color = 0 ORDER BY cartodb_id ASC LIMIT 1);', table_name, table_name) USING current_color;

Go down the list and color every vertex not connected to the colored vertices above the same color

FOR item IN EXECUTE format('SELECT * FROM %s_adjacency_list WHERE color = 0', table_name) LOOP EXECUTE format('SELECT COUNT(*) FROM %s_adjacency_list WHERE color = $1 AND $2 = ANY(adjacent)', table_name) INTO cc USING current_color, item.cartodb_id; IF cc = 0 THEN EXECUTE format('UPDATE %s_adjacency_list SET color = $1 WHERE cartodb_id = $2;', table_name) USING current_color, item.cartodb_id; END IF; END loop;

Finally, repeat on the uncolored vertices with a new color, always working in descending order of valence until all the vertices have been colored.

current_color = current_color + 1; EXECUTE format('SELECT COUNT(*) FROM %s_adjacency_list WHERE color = 0', table_name) into guard;

Now we have two functions that can be stored in our CARTO account by running them into the SQL console in our BUILDER dashboard, but how do we run this map coloring algorithm?

Best option here is using our batch SQL API, this allows us to run any SQL that could take several seconds or minutes safely. In this case we just have to do a SELECT to our greedy function passing the table_name and our user_name. We can do this directly from a terminal:

curl -X POST -H "Content-Type: application/json" -d "{ \"query\": \"SELECT greedy('world_borders', 'aromeu')\" }" https://aromeu.carto.com/api/v2/sql/job?api_key={api_key}

The resulting table world_borders_adjacency_list contains a color assigned for each cartodb_id, now we just can join this table with the original world_borders table and apply a category thematic to visualize the result (plus a bit of CartoCSS magic):

In this case we have colored every adjacent country with a different color, in a total of 5 colors, but can we do it better?

Note: for the sake of simplicity in the examples above and below, we are not working with multipolygons; so that, countries composed of several polygons are assigned different colors to each polygon.

The Kempe’s graph coloring algorithm

In 1879, Alfred B. Kempe tried to prove the 4-color theorem and while years later it was demonstrated that it didn’t solved the problem for all cases, the algorithm he designed still can be used to color the world map using just 4 colors.

The Kempe’s graph color algorithm is as follows:

  • Convert the map to a graph (in this case an adjacency list)
  • Choose a vertex (polygon) with less than five neighbors and remove it from the graph. This may cause some vertices that previously had five or more neighbors to now have less than five.
  • Choose another vertex from the updated graph with less than five neighbors and remove it.
  • Continue until you’ve removed all the vertices from the graph.
  • Add the nodes back the graph in reverse order from which you removed them.
  • Color the added node with a color that is not used by any of its current neighbors.
  • Continue until you’ve colored in the entire graph.

This is a little bit more complex algorithm that still can be solved using a pure PostgreSQL function:

CREATE OR REPLACE FUNCTION kempe(table_name regclass, user_name text) RETURNS void AS $$ DECLARE item RECORD; cc int; count int = 1; d int; temp_cartodb_id int; guard int; BEGIN PERFORM adjacency_list(table_name, user_name); EXECUTE format('DROP TABLE IF EXISTS %s_temp', table_name); EXECUTE format('CREATE TABLE %s_temp (cartodb_id integer, ord integer);', table_name); EXECUTE format('SELECT COUNT(*) FROM %s_adjacency_list WHERE valence < 5 AND valence > 0', table_name) INTO guard; WHILE guard > 0 LOOP EXECUTE format('SELECT cartodb_id FROM %s_adjacency_list WHERE valence < 5 AND valence > 0 ORDER BY valence DESC LIMIT 1', table_name) into temp_cartodb_id; EXECUTE format('INSERT INTO %s_temp (cartodb_id, ord) VALUES ($1, $2)', table_name) USING temp_cartodb_id, count; count = count + 1; EXECUTE format('UPDATE %s_adjacency_list SET valence = 0 WHERE cartodb_id = $1', table_name) USING temp_cartodb_id; FOR item IN EXECUTE format('SELECT * FROM %s_adjacency_list WHERE $1 = ANY(adjacent)', table_name) USING temp_cartodb_id LOOP EXECUTE format('UPDATE %s_adjacency_list SET valence = valence - 1 WHERE cartodb_id = $1', table_name) USING item.cartodb_id; END LOOP; EXECUTE format('SELECT COUNT(*) FROM %s_adjacency_list WHERE valence < 5 AND valence > 0', table_name) INTO guard; END LOOP; EXECUTE format('UPDATE %s_adjacency_list set valence = array_length(adjacent, 1)', table_name); EXECUTE format('UPDATE %s_adjacency_list set color = 0', table_name); FOR item IN EXECUTE format('SELECT * FROM %s_temp ORDER BY ord DESC', table_name) LOOP EXECUTE format('SELECT COUNT(*) FROM %s_adjacency_list WHERE color != 0 AND $1 = ANY(adjacent)', table_name) INTO cc USING item.cartodb_id; IF cc = 0 THEN EXECUTE format('UPDATE %s_adjacency_list SET color = 1 WHERE cartodb_id = $1', table_name) USING item.cartodb_id; ELSE EXECUTE format('select (array_agg(elements))[1] from ( select unnest(array[1, 2, 3, 4, 5]) except SELECT unnest(ARRAY_AGG(color)) FROM %s_adjacency_list WHERE color != 0 AND $1 = ANY(adjacent) ) t (elements)', table_name) INTO d USING item.cartodb_id; EXECUTE format('UPDATE %s_adjacency_list SET color = $1 WHERE cartodb_id = $2', table_name) USING d, item.cartodb_id; END IF; END LOOP; EXECUTE format('DROP TABLE IF EXISTS %s_temp', table_name); END; $$ LANGUAGE plpgsql;

Again we can create the function from the SQL console, execute it for the world_borders dataset using the batch SQL API and then map it with BUILDER. Let’s see the result:

In this case we have colored the world map in 4 colors, challenge accomplished!

Learning point: We have not only learned how to solve the graph coloring problem with CARTO but we have ended up creating the basis for a geospatial framework by creating PostgreSQL functions into our CARTO account.

More map coloring

So, let’s finish by applying these map coloring functions that now are part of our own geospatial framework inside CARTO to some other of our datasets.

The 4 color theorem applied to the US states dataset

A greedy approach to map color the US counties dataset

Another greedy example with the Spain municipalities

Note that the map coloring algorithm implementations presented in this blog post are totally naive and don’t pretend to be exact or to be used under a production environment, they just pretend to showcase a user workflow to solve a geospatial problem with CARTO.

For reference, all these functions are available here. Feel free to add any comment or improve them.

If you like the kind of stuff we are involved in you may want to join us :)

Categories: OSGeo Planet

Stefano Costa: IOSACal 0.4

OSGeo Planet - Fri, 2018-05-11 22:47

IOSACal is an open source program for calibration of radiocarbon dates.

A few days ago I released version 0.4, that can be installed from PyPI or from source. The documentation and website is at http://c14.iosa.it/ as usual. You will need to have Python 3 already installed.

The main highlight of this release are the new classes for summed probability distributions (SPD) and paleodemography, contributed by Mario Gutiérrez-Roig as part of his work for the PALEODEM project at IPHES.

A bug affecting calibrated date ranges extending to the present was corrected.

On the technical side the most notable changes are the following:

  • requires NumPy 1.14, SciPy 1.1 and Matplotlib 2.2
  • removed dependencies on obsolete functions
  • improved the command line interface

You can cite IOSACal in your work with the DOI https://doi.org/10.5281/zenodo.630455. This helps the author and contributors to get some recognition for creating and maintaining this software free for everyone.

Categories: OSGeo Planet

gvSIG Team: Creating new colour ramps in gvSIG Online

OSGeo Planet - Thu, 2018-05-10 13:42

As you knew a raster image could be represented using a ‘colour table’. These colour tables are generated from the colour ramps. In gvSIG Online we have a series of colour ramps but for customized cases that may be interesting to create a new one. There is no problem, gvSIG Online allows us to do it.

We show you how to do it through a video-tutorial. In this video we follow these steps:

  1. We have a geoportal with a raster image where we can apply a concrete symbology (colour table). Let’s imagine that any of the existing colour ramps is what we want to apply.
  2. We access to the dashboard and we see that in our gvSIG Online there are several symbol libraries that have a series of colour ramps.
  3. We add a new symbol library that has two new colour ramps. For each ramp it indicates the colours that configure it.
  4. Finally, we apply a colour table to our raster image, using the new available colour ramps.

As you can see, gvSIG Online is growing in functionalities day by day and it is becoming a reference when implementing Spatial Data Infrastructures.

Categories: OSGeo Planet

gvSIG Team: gvSIG aplicado a Medio Ambiente. Tema 7 (y último): Salida de mapas.

OSGeo Planet - Thu, 2018-05-10 08:13

Finalizamos la parte teórico-practica del curso gratuito gvSIG aplicado a Medio Ambiente aprendiendo a generar mapas imprimibles.


En este tema trabajaremos sobre el “layout”, es decir, el mapa impreso como resultado de todos los análisis y geoprocesos que hemos realizado durante el curso y que realizaremos en nuestro trabajo cotidiano.

Si bien es cierto que la impresión de mapas es cada vez menor desde la aparición de las plataformas de webmapping y las plataformas para la gestión de Infraestructuras de Datos Espaciales (como gvSIG Online) no está de más aprender a manejar esta herramienta para utlizarla en informes técnicos, publicaciones, etc.

Aprenderemos a realizar una plantilla de mapas, a establecer los elementos necesarios para que el mapa sea comprensible y a insertar las vistas de nuestros proyectos para imprimir nuestro trabajo.

Podéis acceder al nuevo tema en el siguiente enlace:


Como siempre, si tenéis dudas, os animamos a escribir en la lista de usuari@s de gvSIG: http://osgeo-org.1560.x6.nabble.com/gvSIG-usuarios-f4118557.html

Categories: OSGeo Planet

Jackie Ng: Announcing: MapGuide Maestro 6.0m9

OSGeo Planet - Wed, 2018-05-09 23:56
Here's a new milestone release of MapGuide Maestro. Here's what's new in this release.

mapguide-react-layout support

One of the design goals of mapguide-react-layout viewer was to be highly compatible with existing Web and Flexible layout documents ensuring for the same authoring experience as the existing AJAX and Fusion viewer offerings.

However, the authoring experience in Maestro knows nothing about mapguide-react-layout, so any authoring experience still assumes the use of AJAX or Fusion viewers.

With this release, we have a new preference for specifying the base URL of a mapguide-react-layout installation.

Once this is set, the Web and Flexible Layout editors light up with additional viewer URLs allowing you to load the Web/Flexible Layout with a mapguide-react-layout template of your choice. 

Since the UI for this has changed from a read-only text box to a combo box, once cannot easily select the URL to copy/paste. To workaround this, a convenience "Copy to Clipboard" button is included to easily copy the current viewer URL for pasting elsewhere.

The other authoring experience change is that a new Flexible Layout will no longer include Fusion widgets that are not supported by mapguide-react-layout. These widgets are really esoteric ones, so most authors won't probably notice any differences.

MgTileSeeder improvements

The new MgTileSeeder tool has been improved with the following changes:
  • A new --failed-requests parameter for specifying a log file to log failed requests to
  • A new --max-parallelism parameter for controlling the max degreee of parallelism when sending tile requests
  • New xyz_replay and mapguide_replay commands for re-requesting failed requests from a log file previously logged via the new --failed-requests parameter
Improved OGR Feature Source support

The OGR provider has been the primary recipient of my continued developer attention since the release of MapGuide Open Source 3.1.1 as there are many things in the provider with room for improvement. Given this, it was time to also give the OGR provider equivalent treatment in Maestro, so with this release, feature sources using the OGR provider now has its own specialized editor.

Most of the UI here should be self-explanatory, but the Other Properties section deserves some explanation. This is a future-proof data grid for editing connection properties that may be introduced in future builds/releases of the OGR provider.

Another case where the OGR provider gets better treatment is the support in the SHP feature source editor to convert it across to use the OGR provider. 

This feature was added to improve the SHP story for MapGuide on 64-bit Linux where the current dedicated SHP provider has unresolved 64-bit portability problems making it unusable. However, the OGR provider has no such issues making it a viable alternative FDO provider for SHP files on 64-bit Linux. This conversion feature helps facilitate this transition in an easy manner. However, there's still some glaring problems with the converted OGR feature source, such as the FDO provider using a hard-coded schema name of "OGRSchema". Nothing some python scripting elbow grease can't fix afterwards. Or you can watch this space :)

Other fixes/changes
  • Fix: Rename resource with "update references" checked will disregard overwrite flag
  • Fix: NullReferenceException when ticking a new geometry type and adding a rule to the grid for the first time in Layer Definition editor
  • Fix: MgInvalidRepositoryTypeException when validating layers. Previous workaround was to disable validation on save. With this release you can safely re-enable this if you so choose.
  • Fix: Editing default path in line usage context in Symbol Definition editor does nothing
  • Fix: Cannot browse symbol definiton parameters in any field of the Path editor dialog 
Project Web Site
Categories: OSGeo Planet
Syndicate content