OSGeo Planet

Fernando Quadro: Divisão de polígonos no PostGIS

OSGeo Planet - Thu, 2018-09-06 14:15

Uma das coisas interessantes do processamento geoespacial é a variedade de ferramentas, e as maneiras de colocá-las juntas podem produzir resultados surpreendentes.

Um membro da comunidade na lista de usuários do PostGIS perguntou: “Existe uma maneira de dividir um polígono em subpolígonos de áreas mais ou menos iguais?”

O desenvolvedor do PostGIS, Darafei Praliaskouski, respondeu e forneceu uma solução funcional que é absolutamente brilhante ao combinar as partes do kit de ferramentas do PostGIS para resolver um problema bastante complicado. Ele disse:

Do jeito que eu vejo, para qualquer tipo de polígono:

  • Converte um polígono em um conjunto de pontos proporcionais à área por ST_GeneratePoints (quanto mais pontos, mais bonito será, acho que 1000 está ok);
  • Decida quantas partes você gostaria de dividir em (ST_Area (geom) / max_area), seja K;
  • Para cada cluster, pegue um ST_Centroid (ST_Collect (point));
  • Alimente esses centróides em ST_VoronoiPolygons, que lhe dará uma máscara para cada parte do polígono;
  • Com o ST_Intersection do polígono original e cada célula dos polígonos de Voronoi você obterá uma boa divisão do seu polígono em K partes.

Vamos dar um passo de cada vez para ver como funciona.

Usaremos o Peru como polígono de exemplo, ele tem uma boa concavidade, o que o torna um pouco mais complicado do que um polígono com “forma comum”.

CREATE TABLE peru AS SELECT * FROM countries WHERE name = 'Peru'

Agora crie um campo de pontos que preencha o polígono. Em média, cada ponto colocado aleatoriamente acaba “ocupando” uma área igual dentro do polígono.

CREATE TABLE peru_pts AS SELECT (ST_Dump(ST_GeneratePoints(geom, 2000))).geom AS geom FROM peru WHERE name = 'Peru'

Agora, agrupe o campo de pontos, definindo o número de clusters para o número de partes em que você deseja dividir o polígono. Visualmente, agora você pode ver as divisões no polígono! Mas ainda precisamos obter linhas reais para representar essas divisões.

CREATE TABLE peru_pts_clustered AS SELECT geom, ST_ClusterKMmeans(geom, 10) over () AS cluster FROM peru_pts;

Primeiro, calcule o centroide de cada cluster de pontos, que será o centro de massa de cada cluster.

CREATE TABLE peru_centers AS SELECT cluster, ST_Centroid(ST_collect(geom)) AS geom FROM peru_pts_clustered GROUP BY cluster;

Agora, use um diagrama de voronoi para obter arestas de divisão reais entre os centróides do cluster, que acabam combinando de perto com os locais onde os clusters se dividem!

CREATE TABLE peru_voronoi AS SELECT (ST_Dump(ST_VoronoiPolygons(ST_collect(geom)))).geom AS geom FROM peru_centers;

Finalmente, intercepte as áreas de voronoi com o polígono original para obter os polígonos de saída final que incorporam as bordas externas das linhas de divisão.

CREATE TABLE peru_divided AS SELECT ST_Intersection(a.geom, b.geom) AS geom FROM peru a CROSS JOIN peru_voronoi b;

Feito!

Agrupar um campo de pontos para obter áreas praticamente iguais e, em seguida, usar o voronoi para extrair linhas divisórias reais são insights maravilhosos sobre o processamento espacial. A imagem final de todos os componentes do cálculo também é bonita.

Não tenho 100% de certeza, mas talvez seja possível usar a técnica de Darafei para subdivisões ainda mais interessantes, como “mapa do Brasil subdividido em áreas de igual PIB”, ou “mapa de São Paulo subdividido em áreas de igual tamanho”, ou ainda a população ”gerando o campo de ponto inicial usando uma ponderação econômica ou demográfica”.

Este texto foi traduzido e adaptado do post original de Paul Ramsey no blog Clever Elephant.

Fonte: Blog Clever Elephant

Categories: OSGeo Planet

geomati.co: Querying multiple layers in a single query with GeoServer?

OSGeo Planet - Thu, 2018-09-06 07:11

Our problem, get multiple features filtered from multiple different layers using WFS.

Let’s work with the topp data coming by default in GeoServer. A matter of taste.

To apply a single CQL_filter on a WFS layer it’s simple. We have the “unknown” ;-) topp:tasmania_water_bodies published in one Demo GeoServer (thanks GeoSolutions ;-)) and we want to get the features where the area is greater than 1066494066 square meters, so we can make the next petition to the server

https://demo.geo-solutions.it/geoserver/wfs?SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&typeNames=topp:tasmania_water_bodies&propertyName=&cql_filter=AREA<1066494066&outputFormat=application/json

Easy! We’ll get 4 features that satisfy filters. The syntax is clear, we must use typeNames=topp:tasmania_water_bodies and cql_filter=AREA<1066494066 in the petition.

But, how we must do if we want get features from two or more layers at the same time?.

If we get at the same time the features type Alley from the topp:tasmania_roads layer with the topp:tasmania_water_bodies features filtered before, we must use similar request but separating the typeNames in this way:

typeNames=(topp:tasmania_water_bodies)(topp:tasmania_roads)

and setting the associated CQL filters ordered respect the typenames:

cql_filter=AREA<1066494066;TYPE='alley'

https://demo.geo-solutions.it/geoserver/wfs?SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&typeNames=(topp:tasmania_water_bodies)(topp:tasmania_roads)&propertyName=&cql_filter=AREA<1066494066;TYPE=‘alley’&outputFormat=application/json

And, what about the propertyName?. Using the propertyName parameter on the WFS petition we can filter the returned feature properties as we want. This parameter is important to reduce the response’s size getting only the properties we are interested in.

In our example, we can get only the CNTRY_NAME of the topp:tasmania_water_bodies and the TYPE of the topp:tasmania_roads.

In this case the request will be:

https://demo.geo-solutions.it/geoserver/wfs?SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&typeNames=(topp:tasmania_water_bodies)(topp:tasmania_roads)&propertyName=(CNTRY_NAME)(TYPE)&cql_filter=AREA%3C1066494066;TYPE=%27alley%27&outputFormat=application/json

Michogar’s GIST

Enjoy!!.

Categories: OSGeo Planet

gvSIG Team: Taller sobre gvSIG Mobile en las 14as Jornadas Internacionales gvSIG

OSGeo Planet - Wed, 2018-09-05 14:45

Durante las 14as Jornadas Internacionales gvSIG, que tendrán lugar en Valencia (España) del 24 al 26 de octubre, se realizará un taller sobre gvSIG Mobile, el Sistema de Información Geográfica en software libre para dispositivos móviles que permite la toma de datos en campo, muy útil para censos, inventarios, inspecciones, encuestas…

Durante el mismo los participantes aprenderán a crear un fichero de teselas a partir de una imagen georreferenciada en gvSIG Desktop, el cual puede ser insertado como capa base en gvSIG Mobile, evitando así la necesidad de disponer de conexión a internet para tener una capa base.

Por otro lado, se mostrará cómo crear formularios personalizados en gvSIG Desktop que serán exportados posteriormente a gvSIG Mobile para poder tomar datos en campo, pudiendo crear campos de diferentes tipos (texto, fecha, desplegables…) o realizar fotografías o croquis.

Respecto a las herramientas de edición, durante el taller se creará una capa vectorial en gvSIG Desktop que posteriormente será editada en gvSIG Mobile, tanto gráfica como alfanuméricamente.

También se verán otras herramientas como la importación de marcadores, carga de capas WMS…

En las próximas semanas se facilitará en este mismo post toda la información sobre cómo inscribirse en este taller, que será gratuito, así como la cartografía a descargar para poder seguirlo.

En unos días se dará más información también sobre el día y hora en que se impartirá.

Si deseas asistir a las jornadas, te recordamos que el periodo de inscripción a las mismas ya está abierto, pudiendo realizarse el registro a través del formulario habilitado para ello.

¡No os perdáis este taller!

Categories: OSGeo Planet

Free and Open Source GIS Ramblings: Plotting GPS Trajectories with error ellipses using Time Manager

OSGeo Planet - Tue, 2018-09-04 17:54

This is a guest post by Time Manager collaborator and Python expert, Ariadni-Karolina Alexiou.

Today we’re going to look at how to visualize the error bounds of a GPS trace in time. The goal is to do an in-depth visual exploration using QGIS and Time Manager in order to learn more about the data we have.

The Data

We have a file that contains GPS locations of an object in time, which has been created by a GPS tracker. The tracker also keeps track of the error covariance matrix for each point in time, that is, what confidence it has in the measurements it gives. Here is what the file looks like:

data.png

Error Covariance Matrix

What are those sd* fields? According to the manual: The estimated standard deviations of the solution assuming a priori error model and error parameters by the positioning options. What it basically means is that the real GPS location will be located no further than three standard deviations across north and east from the measured location, most of (99.7%) the time. A way to represent this visually is to create an ellipse that maps this area of where the real location can be.ellipse_ab

An ellipse can be uniquely defined from the lengths of the segments a and b and its rotation angle. For more details on how to get those ellipse parameters from the covariance matrix, please see the footnote.

Ground truth data

We also happen to have a file with the actual locations (also in longitudes and latitudes) of the object for the same time frame as the GPS (also in seconds), provided through another tracking method which is more accurate in this case.

actual_data

This is because, the object was me running on a rooftop in Zürich wearing several tracking devices (not just GPS), and I knew exactly which floor tiles I was hitting.

The goal is to explore, visually, the relationship between the GPS data and the actual locations in time. I hope to get an idea of the accuracy, and what can influence it.

First look

Loading the GPS data into QGIS and Time Manager, we can indeed see the GPS locations vis-a-vis the actual locations in time.

actual_vs_gps

Let’s see if the actual locations that were measured independently fall inside the ellipse coverage area. To do this, we need to use the covariance data to render ellipses.

Creating the ellipses

I considered using the ellipses marker from QGIS.

ellipse_marker.png

It is possible to switch from Millimeter to Map Unit and edit a data defined override for symbol width, height and rotation. Symbol width would be the a parameter of the ellipse, symbol height the b parameter and rotation simply the angle. The thing is, we haven’t computed any of these values yet, we just have the error covariance values in our dataset.

Because of the re-projections and matrix calculations inherent into extracting the a, b and angle of the error ellipse at each point in time, I decided to do this calculation offline using Python and relevant libraries, and then simply add a WKT text field with a polygon representation of the ellipse to the file I had. That way, the augmented data could be re-used outside QGIS, for example, to visualize using Leaflet or similar. I could have done a hybrid solution, where I calculated a, b and the angle offline, and then used the dynamic rendering capabilities of QGIS, as well.

I also decided to dump the csv into an sqlite database with an index on the time column, to make time range queries (which Time Manager does) run faster.

Putting it all together

The code for transforming the initial GPS data csv file into an sqlite database can be found in my github along with a small sample of the file containing the GPS data.

I created three ellipses per timestamp, to represent the three standard deviations. Opening QGIS (I used version: 2.12, Las Palmas) and going to Layer>Add Layer>Add SpatialLite Layer, we see the following dialog:

add_spatialite2.png

After adding the layer (say, for the second standard deviation ellipse), we can add it to Time Manager like so:

add_to_tm

We do the process three times to add the three types of ellipses, taking care to style each ellipse differently. I used transparent fill for the second and third standard deviation ellipses.

I also added the data of my  actual positions.

Here is an exported video of the trace (at a place in time where I go forward, backwards and forward again and then stay still).

gps

Conclusions

Looking at the relationship between the actual data and the GPS data, we can see the following:

  • Although the actual position differs from the measured one, the actual position always lies within one or two standard deviations of the measured position (so, inside the purple and golden ellipses).
  • The direction of movement has greater uncertainty (the ellipse is elongated across the line I am running on).
  • When I am standing still, the GPS position is still moving, and unfortunately does not converge to my actual stationary position, but drifts. More research is needed regarding what happens with the GPS data when the tracker is actually still.
  • The GPS position doesn’t jump erratically, which can be good, however, it seems to have trouble ‘catching up’ with the actual position. This means if we’re looking to measure velocity in particular, the GPS tracker might underestimate that.

These findings are empirical, since they are extracted from a single visualization, but we have already learned some new things. We have some new ideas for what questions to ask on a large scale in the data, what additional experiments to run in the future and what limitations we may need to be aware of.

Thanks for reading!

Footnote: Error Covariance Matrix calculations

The error covariance matrix is (according to the definitions of the sd* columns in the manual):

sde * sde sign(sdne) * sdne * sdne sign(sdne) * sdne * sdne sdn * sdn

It is not a diagonal matrix, which means that the errors across the ‘north’ dimension and the ‘east’ dimension, are not exactly independent.

An important detail is that, while the position is given in longitudes and latitudes, the sdn, sde and sdne fields are in meters. To address this in the code, we convert the longitude and latitudes using UTM projection, so that they are also in meters (northings and eastings).

For more details on the mathematics used to plot the ellipses check out this article by Robert Eisele and the implementation of the ellipse calculations on my github.

Categories: OSGeo Planet

Paul Ramsey: Moving on to Crunchy Data

OSGeo Planet - Tue, 2018-09-04 16:00

Today is my first day with my new employer Crunchy Data. Haven’t heard of them? I’m not surprised: outside of the world of PostgreSQL, they are not particularly well known, yet.

Moving on to Crunchy Data

I’m leaving behind a pretty awesome gig at CARTO, and some fabulous co-workers. Why do such a thing?

While CARTO is turning in constant growth and finding good traction with some core spatial intelligence use cases, the path to success is leading them into solving problems of increasing specificity. Logistics optimization, siting, market evaluation.

Moving to Crunchy Data means transitioning from being the database guy (boring!) in a geospatial intelligence company, to being the geospatial guy (super cool!) in a database company. Without changing anything about myself, I get to be the most interesting guy in the room! What could be better than that?

Crunchy Data has quietly assembled an exceptionally deep team of PostgreSQL community members: Tom Lane, Stephen Frost, Joe Conway, Peter Geoghegan, Dave Cramer, David Steele, and Jonathan Katz are all names that will be familiar to followers of the PostgreSQL mailing lists.

They’ve also quietly assembled expertise in key areas of interest to large enterprises: security deployment details (STIGs, RLS, Common Criteria); Kubernetes and PaaS deployments; and now (ta da!) geospatial.

Why does this matter? Because the database world is at a technological inflection point.

Core enterprise systems change very infrequently, and only under pressure from multiple sources. The last major inflection point was around the early 2000s, when the fleet of enterprise proprietary UNIX systems came under pressure from multiple sources:

  • The RISC architecture began to fall noticeably behind x86 and particular x86-64.
  • Pricing on RISC systems began to diverge sharply from x86 systems.
  • A compatible UNIX operating system (Linux) was available on the alternative architecture.
  • A credible support company (Red Hat) was available and speaking the language of the enterprise.

The timeline of the Linux tidal wave was (extremely roughly):

  • 90s - Linux becomes the choice of the tech cognoscenti.
  • 00s - Linux becomes the choice of everyone for greenfield applications.
  • 10s - Linux becomes the choice of everyone for all things.

By my reckoning, PostgreSQL is on the verge of a Linux-like tidal wave that washes away much of the enterprise proprietary database market (aka Oracle DBMS). Bear in mind, these things pan out over 30 year timelines, but:

  • Oracle DBMS offers no important feature differentiation for most workloads.
  • Oracle DBMS price hikes are driving customers to distraction.
  • Server-in-a-cold-room architectures are being replaced with the cloud.
  • PostgreSQL in the cloud, deployed as PaaS or otherwise, is mature.
  • A credible support industry (including Crunchy Data) is at hand to support migrators.

I’d say we’re about half way through the evolution of PostgreSQL from “that cool database” to “the database”, but the next decade of change is going to be the one people notice. People didn’t notice Linux until it was already running practically everything, from web servers to airplane seatback entertainment systems. The same thing will obtain in database land; people won’t recognize the inevitability of PostgreSQL as the “default database” until the game is long over.

Having a chance to be a part of that change, and to promote geospatial as a key technology while it happens, is exciting to me, so I’m looking forward to my new role at Crunchy Data a great deal!

Meanwhile, I’m going to be staying on as a strategic advisor to CARTO on geospatial and database affairs, so I get to have a front seat on their continued growth too. Thanks to CARTO for three great years, I enjoyed them immensely!

Categories: OSGeo Planet

MapTiler: Design own map with five clicks

OSGeo Planet - Tue, 2018-09-04 11:00
Thumb

Creating beautiful map fitting your brand is no longer a difficult job. With MapTiler Cloud, you pick five colors and the entire map adjust accordingly.

Own map design straightforward

If you ever browsed our maps, maybe you got the feeling: this style is almost perfect but I need just a small change to fit the map with app or web design. Now you can fix it only with a few mouse clicks.

The Make your own map button will lead you directly to the Customize tool where you can drag just one color and all map elements in the group adjust accordingly. The changes can be reset to the default if needed so there is no need to hold back during experimenting. You can also copy a color scheme from a map you made before, change the language of labels to one of 55+ supported languages or the font.

The map design you create in Customize tool will automatically stay updated when we update the software running behind or add a new type of data.

Once you are happy with your design, you can log in or create a free account to save your changes and directly use it on your website or in the app.

Intelligent Maps: the magic behind

MapTiler Cloud offers more than just creating own design. Intelligent Maps can dynamically adapt colors, languages and data for individual visitors and perfectly fit the need of companies who use the maps in their web or mobile applications.

Your visitors can get a personalized map in the applications, with place names automatically displayed in their language, with highlights of most important places for each individual end-user - such as most often used bus stops, frequently visited restaurants or recent trips. It can be done based on the user’s preferences, his history, or data loaded from existing information systems. The maps can also change based on the day of the week, daytime and opening hours, so the user receives only the relevant information.

Tokyo in Customize tool

Free to try, no registration required

To create own map design, you need no registration. Simply visit our maps and start customizing your favorite one. After creating a free account, you can save your map and directly start using it in your web or app.

Categories: OSGeo Planet

SourcePole: FOSS4G 2018 Dar es Salaam

OSGeo Planet - Tue, 2018-09-04 00:00

This year’s FOSS4G edition took place in Dar es Salaam, Tanzania. As every year, Sourcepole was supporting this major event as a sponsor. We would like to thank for all the interesting discussions and feedback to our presentations!

image

QGIS Web Client 2 Update Styling and publication of vector tiles Using GeoPackage as work and exchange format

Thanks to the LOC for organizing another great FOSS4G!

Pirmin Kalberer (@implgeo)

Categories: OSGeo Planet

GeoSolutions: GeoSolutions presentations from FOSS4G 2018!

OSGeo Planet - Sun, 2018-09-02 18:06

FOSS4G 2018

Dear Reader,

we are putting together in this post all presentations that were given by our staff during this year FOSS4G in Dar Es Salam, Tanzania.

Here is the complete list. Enjoy!

State of GeoServer 2018 Serving earth observation data with GeoServer: addressing real world requirements Creating Stunning Maps in GeoServer, with SLD, CSS, YSLD and MBStyles Crunching Data In GeoServer: Mastering Rendering Transformations, WPS And SQL Views GeoServer in production. We do it, here is how! One GeoNode, many GeoNodesMapping beyond web mercatorMapStore 2, the story

If you want further information, do not hesitate to contact us.

The GeoSolutions Team,

320x100_eng
Categories: OSGeo Planet

Paul Ramsey: BC IT Outsourcing 2017/18

OSGeo Planet - Thu, 2018-08-30 07:00

A year has gone by and the public accounts are out again, so I have updated my running tally of IT outsourcing for the BC government.

While the overall growth is modest, I’m less sure of my numbers this year because of last year’s odd and sudden drop-off of the IBM amount.

I am guessing that the IBM drop should be associated with their loss of desktop support services for the health authorities, which went to Fujitsu, but for some reason the transfer is not reflected in the General Revenue accounts. The likely reason is that it’s now being paid out of some other bucket, like the Provincial Health Services Authority (PHSA), so fixing the trend line will involve finding that spend in other accounts. Unfortunately, Health Authorities won’t release their detailed accounts for another few months.

All this shows the difficulty in tracking IT spend over the whole of government, which is something the Auditor General remarked on when she reviewed IT spending a few years ago. Capital dollars are relatively easy to track, but operating dollars are not, and of course the spend is spread out over multiple authorities as well as in general revenue.

In terms of vendors, the drop off in HP/ESIT is interesting. For those following at home ESIT (formerly known as HP Advanced Solutions) holds the “back-office” contract, which means running the two government data centres (in Calgary and Kamloops (yes, the primary BC data centre is in Alberta)) as well as all the servers therein and the networks connecting them up. It’s a pretty critical contract, and the largest in government. Since being let (originally to Ross Perot’s EDS), one of the constants of this tracking has been that the amount always goes up. So this reduction is an interesting data point: will it hold?

And there’s still a large whale unaccounted for: the Coastal Health Authority electronic health record project, which has more-or-less dropped off the radar since Adrian Dix brought in Wynne Powell as a fixer. The vendors (probably Cerner and others) for that project appear on neither the PHSA nor Coastal Health public accounts, I think because it is all being run underneath a separate entity. I haven’t had a chance to figure out the name of it (if you know the way this is financed, drop me a line).

Categories: OSGeo Planet

Even Rouault: SRS barn raising: 3rd report

OSGeo Planet - Wed, 2018-08-29 14:23
This is the third progress report of the GDAL SRS barn effort.

In the last month, the main task was to continue, and finish, the mapping of all GDAL currently supported projection methods between their different representations as WKT 2, WKT 1 and PROJ string: LCC_2SP, LCC_2SP_Belgium, Modified Azimuthal Equidistant, Guam Projection, Bonne, (Lambert) Cylindrical Equal Area,    GaussSchreiberTransverseMercator, CassiniSoldner, EckertI to VI, EquidistantCylindricalSpherical, Gall, GoodeHomolosine, InterruptedGoodeHomolosine, GeostationarySatelliteSweepX/Y, International Map of the World Polyconic, Krovak North Oriented and Krovak, LAEA, Mercator_1SP and Mercator_2SP, WebMercator (including GDAL WKT1 import/export tricks), Mollweide, ObliqueStereographic and Orthographic, American polyconic, PolarSterographicVariantA and PolarSterographicVariantB, Robinson and Sinusoidal, Stereographic, VanDerGrinten, Wagner I to VII, QuadrilateralizedSphericalCube, SphericalCrossTrackHeight, Aitoff, Winkel I and II, Winkel Tripel, Craster_Parabolic, Loximuthal and Quartic_Authalic
The task was tedious, but necessary.  For some cases, this involved cross-checking formulas in the EPSG "Guidance Note 7, part 2 Coordinate Conversions & Transformations including Formulas", PROJ implementation and Snyder "Map Projections - A Working Manual" because of ambiguities in some projection names. Typically the ObliqueStereographic in EPSG is not the Oblique Stereographic of Snyder. The former is implemented as the Oblique Stereographic Alternative (sterea) in PROJ, and the later as the Oblique Stereographic (stere). The parameter names in WKT 2 / EPSG tend also to be much more specific that in GDAL WKT 1. When in GDAL WKT1, you have mostly a "latitude_of_origin" parameter mapping to the lat_0 PROJ parameter, in WKT2, parameter names tend to better reflect the mathematical characteristics of the projection, distinguishing between "Latitude of natural origin", "Latitude of projection centre" or "Latitude of false origin"
The currently ongoing task is now to implement a method that takes a PROJ string and builds ISO-19111 corresponding objects. Done for GeographicCRS (+proj=longlat), in progress for Projected CRS. When this will be completed we will have the infrastructure to convert in all directions between PROJ strings, WKT 1 and WKT 2
When digging into PROJ code, I also uncovered a number of issues in the Helmert implementation (confusing for rotational parameters regarding the "Position Vector" vs "Coordinate frame" convention), the handling of the not-so-well-known +geoc flag for geocentric latitudes and the handling of vertical units for geographic CRS with the new PROJ API. All of those fixes have been independantly merged in PROJ master, so as to be available for the upcoming PROJ 5.2.0, which should be released around mid-september (to remove any confusion, this release will not include yet all the WKT 2 related work)
Categories: OSGeo Planet

gvSIG Team: JIIDE 2018: Asociación gvSIG en las Jornadas Ibéricas de Infraestructuras de Datos Espaciales

OSGeo Planet - Tue, 2018-08-28 11:30

El texto que acompaña la presentación de las JIIDE de este año dice así:

En estos momentos en los que los mayores desafíos a los que se enfrenta la humanidad, como son el cambio climático, una economía sostenible, el reparto justo de la riqueza, la degradación del medio ambiente, las migraciones y en general, los efectos negativos de la globalización, tienen un marcado carácter geoespacial, las IDE, como sistemas de sistemas colaborativos, abiertos e interoperables son más necesarios que nunca.

A lo que debemos añadir: libres. Sistemas libres, de código abierto, que permitan reutilizar, sumar y apostar por un modelo económico diferente, sostenible, alejado de monopolios y que evite caer en los riesgos de la dependencia tecnológica o de proveedores únicos.

En la suma de los dos textos encontramos la visión y misión de la Asociación gvSIG, que en estas JIIDE 2018 presentará un taller y una serie de ponencias relacionadas con la Suite gvSIG:

  • 17 de octubre.

    • Sesión IDE locales. Sala 1 [11:30-13:30]. Infraestructuras de Datos Espaciales municipales con gvSIG Suite

  • 18 de octubre.

    • Sesión Interoperabilidad de datos y servicios geográficos. Sala 2 [09:00-11:00]. IDE para bomberos, protección civil y gestión del delito

    • Sesión IDE nacionales, regionales y locales II. Sala 1 [12:30-14:30]. IDE en software libre para Agricultura: el caso de gvSIG Online en la GVA

    • Taller 4 [16:00-17:00]. gvSIG Online, plataforma en software libre para IDE

Las JIIDE 2018 tendrán lugar los días 17, 18 y 19 de octubre en la Isla del Lazareto, en el puerto de Maó. Estaremos encantados de interoperar e intercambiar conocimiento.

Categories: OSGeo Planet

GeoServer Team: GeoServer 2.14-RC released

OSGeo Planet - Tue, 2018-08-28 11:05

We are happy to announce the release of GeoServer 2.14-RC. Downloads are available (zipwar, and exe) along with docs and extensions.

This is a beta release of GeoServer made in conjunction with GeoTools 20-RC.

We want to encourage people to test the release thoroughly and report back any issue found. With no further delay, let’s see what’s new, that is, what is there to test!

WMS “nearest match” support for time dimension

WMS time dimension can now be configured for “nearest match”, that is, to return the nearest time to the one requested (either explicitly, or implicitly via the default time value).

In case of mismatch the actual time used will be returned along with the response as a HTTP header. It is also possible to configure a maximum tolerance, beyond that the server will throw a service exception.

Channel selection name allow expressions

GeoServer 2.14-RC allows expressions to be used in SourceChannelName SLD elements, and in their code counterpart, thus allowing dynamic channel selection. This is welcomed news for anyone building applications that display multispectral or hyperspectral data, thus avoiding the need to build many different styles for the various interesting false color combinations.

Here is an SLD example:

<RasterSymbolizer> <ChannelSelection> <RedChannel> <SourceChannelName> <ogc:Function name="env"> <ogc:Literal>B1</ogc:Literal> <ogc:Literal>2</ogc:Literal> </ogc:Function> </SourceChannelName> </RedChannel> <GreenChannel> <SourceChannelName> <ogc:Function name="env"> <ogc:Literal>B2</ogc:Literal> <ogc:Literal>5</ogc:Literal> </ogc:Function> </SourceChannelName> </GreenChannel> <BlueChannel> <SourceChannelName> <ogc:Function name="env"> <ogc:Literal>B3</ogc:Literal> <ogc:Literal>7</ogc:Literal> </ogc:Function> </SourceChannelName> </BlueChannel> </ChannelSelection> <RasterSymbolizer> Map algebra

This release adds support for an efficient map algebra package knows as Jiffle. Jiffle has been the work of a former GeoTools contributor, Michael Bedwards, that has been salvaged, upgraded to support Java 8, and integrated in jai-ext. From the there support has been added into the GeoTools gt-process-raster module and as a result in GeoServer WPS, to be used either directly or as a rendering transformation.

The following SLD style calls onto Jiffle to perform a NDVI calculation on top of Sentinel 2 data:

<?xml version="1.0" encoding="UTF-8"?> <StyledLayerDescriptor xmlns="http://www.opengis.net/sld" xmlns:ogc="http://www.opengis.net/ogc" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/sld http://schemas.opengis.net/sld/1.0.0/StyledLayerDescriptor.xsd" version="1.0.0"> <NamedLayer> <Name>Sentinel2 NDVI</Name> <UserStyle> <Title>NDVI</Title> <FeatureTypeStyle> <Transformation> <ogc:Function name="ras:Jiffle"> <ogc:Function name="parameter"> <ogc:Literal>coverage</ogc:Literal> </ogc:Function> <ogc:Function name="parameter"> <ogc:Literal>script</ogc:Literal> <ogc:Literal> nir = src[7]; vir = src[3]; dest = (nir - vir) / (nir + vir); </ogc:Literal> </ogc:Function> </ogc:Function> </Transformation> <Rule> <RasterSymbolizer> <Opacity>1.0</Opacity> <ColorMap> <ColorMapEntry color="#000000" quantity="-1"/> <ColorMapEntry color="#0000ff" quantity="-0.75"/> <ColorMapEntry color="#ff00ff" quantity="-0.25"/> <ColorMapEntry color="#ff0000" quantity="0"/> <ColorMapEntry color="#ffff00" quantity="0.5"/> <ColorMapEntry color="#00ff00" quantity="1"/> </ColorMap> </RasterSymbolizer> </Rule> </FeatureTypeStyle> </UserStyle> </NamedLayer> </StyledLayerDescriptor>

The performance is good enough for interactive display, and the result looks as follows (click to enlarge):

PostGIS store improvements and measured geometries support

The PostGIS datastore has been for years the only one that could encode a few filter functions used in filters down into native SQL, but it required a datastore creation flag to be enabled.
Starting with this release it will do so by default.

The functions supported for SQL encoding by the store are:

  • String functions: strConcat, strEndsWith, strStartsWith, strEqualsIgnoreCase, strIndexOf, strLength, strToLowerCase, strToUpperCase, strReplace, strSubstring, strSubstringStart, strTrim, strTrim2
  • Math functions: abs, abs_2, abs_3, abs_4, ceil, floor
  • Date functions: dateDifference

This release adds support for “array” data type in the store, with full reading and writing support, as well as native filtering (with index support, where feasible).

Finally, it’s now possible to read geometries with measures from PostGIS and encode the results in GML. GML does not natively support measures, so the encoding is off by default and you’ll have to enable it explicitly, as well as ensure that the clients involved in WFS usage recognize this extra ordinate. The work will continue in the next few month in order to cover more formats.

 

Image mosaic improvements

The image mosaic module never sleeps, in this release we see the following improvements:

  • Support for remote images (e.g. S3 or Minio). In order to leverage this the mosaic index will have to be created up-front (manually, or with some home grown tools)
  • A new “virtual native resolution” read parameter allows the mosaic to compose outputs respecting a native resolution other than its native one (useful in case you want to give selective resolution access to different users)
  • Supporting multiple WKBs footprint for pixel precise overviews masking
  • A new read mask parameter allows to cut the image to a given geometry (again, helps in providing different selective views to different users)
  • Speed up NetCDF mosaics by allowing usage of stores coming from a repository, instead of re-creating them every time a NetCDF file is needed (to be setup in the auxiliary store config file, while the repository instance needs to be passed as a creation hint).
  • The mosaic now works against images without a proper CRS, setting them into the “CRS not found” wildcard CRS, aka “EPSG:404000”
App-schema improvements

The app-schema module got significant performance and functionality improvements since 19.x series, in particular:

  • Better delegation of spatial filters on nested properties to native database
  • Improved support for multiple nested attribute matches in filters
  • It is now possible to use Apache SOLR as a data source for app-schema
  • The configuration files can be hosted on a remote HTTP server
The MongoDB store upgrades to official extension

Wanted to use the MongoDB store but worried about its unsupported status? Worry no more, in GeoServer 2.14 the MongoDB datastore upgraded to official extension status.

Style editor improvements

The GeoServer style editor now includes a fullscreen side-by-side editing mode, to make it easier to preview your styles while editing them. Click the fullscreen button at the top-right of the screen to toggle fullscreen mode.

The toolbar also has two new additions, a color picker helping to find the right color and turn it into a HEX specification, and a file chooser that allows to pick an external graphic and build the relevant ExternalGraphic element:

 

Windows build restored

GeoServer failed to properly build on Windows for a long time. GeoServer 2.14.x is the first branch in years to successfully build on Windows, and we have added an AppVeyor build to help keep the build going in the future.

The work to make it build there has been fully done in spare time, and we are still experiencing random build failures. If you are a Java developer on Windows, we could really use your help to keep GeoServer Windows build friendly.

New community modules

The 2.12 series comes with a few new community modules, in particular:

  • OAuth2 OpenID connect module (still very much a work in progress)
  • A WFS3 implementation has been added that matches the current WFS3 specification draft, and it’s being updated to become an official compliant one.

Mind, community modules are not part of the release, but you can find them in the nightly builds instead.

Other assorted improvements

There are many improvements to look at in the release notes, cherry picking a few here:

  • The ncWMS community module has seen significant performance improvements
  • WPS has a CSV input/output supporting geometryless data, as well as geometries with WKT encoding, as well as supporting pure PNG/JPEG image for raster data
  • Time/Elevation parsers are no longer silently cutting the read data to 100 entries
  • The WMTS multidimensional GetHistogram operation is now significantly faster, and a new GetDomainValues operation allows to page through the values of a domain (time, elevation) that has too many values to be efficiently explored with DescribeDomain or GetFeature. The DescribeDomain was also improved to allow a selection of the domains that should be described.
  • The SLDService community module has now the ability to return a full SLD style (as opposed to snippets), allows for custom classification and custom color selection. Also, keep an eye on this module, as it’s about to graduate to supported status
  • The monitoring modul can now log the hit/miss status of tiled requests, quite helpful to verify the benefit of caching, especially while using WMS direct integration
Security consideration

Please update your production instances of GeoServer to receive the latest security updates and fixes.

This release addresses several security vulnerabilities:

  • Prevent arbitrary code execution via Freemarker Template injection
  • XXE vulnerability in GeoTools XML Parser
  • XXE vulnerability in WPS Request builder
  • Various library upgrades (see above) from versions with known CVEs
  • Potential access to admin pages without being logged in

Thanks to Steve Ikeoka, Kevin Smith, Brad Hards, Nuno Oliveira and Andrea Aime for providing fixes to these issues.

If you encounter a security vulnerability in GeoServer, or any other open source software, please take care to report the issue in a responsible fashion.

Test, test, test!

Now that you know about all the goodies, please go, download and test your favourite ones. Let us know how it went!

About GeoServer 2.14

GeoServer 2.14 is scheduled for September 2018 release.

Categories: OSGeo Planet

GeoServer Team: FOSS4G 2018 GeoServer Developers Workshop

OSGeo Planet - Mon, 2018-08-27 14:28

Thanks to everyone who attended today’s foss4g developer workshop, here are the slides and resources used:

 

Categories: OSGeo Planet

GeoSolutions: Meet GeoSolutions at INSPIRE Conference 2018

OSGeo Planet - Mon, 2018-08-27 14:11

INSPIRE Conference 2018

Dear All, we are proud to announce that GeoSolutions is exhibiting at the INSPIRE Conference 2018 which will be held in Antwerp from 18th to 21st of September 2018.

GeoSolutions will be present at the exhibition at the OSGEO booth, therefore we will be happy to talk to you about our open source products, like GeoServerMapstore, GeoNode and GeoNetwork, as well as about our Enterprise Support Services and GeoServer Deployment Warranty offerings.

Our INSPIRE & GeoServer expert Nuno Oliveira together with our direction Simone Giannecchini are going to hold a workshop on GeoServer for INSPIRE, here is the details:

and will participate to a more general workshop as follows:

as well as a a presentation on the same topic as follows:

If you are interested in learning about how we can help you achieving your goals with our Open Source products and professional services, make sure to visit us at our booths S4-B3-B11 (we are a large family so we took many booths :) ).

See you in Antwerpen!

The GeoSolutions team,
Categories: OSGeo Planet

GeoServer Team: GeoServer 2.12.5 released

OSGeo Planet - Fri, 2018-08-24 19:52

We are happy to announce the release of GeoServer 2.12.5. Downloads are available (zipwar, and exe) along with docs and extensions.

This is the last maintenance release for the 2.12.x series, so we recommend users to plan an upgrade to 2.13.x or to the upcoming 2.14.x series. This release is made in conjunction with GeoTools 18.5.

Highlights of this release are featured below, for more information please see the release notes (2.12.52.12.42.12.3,2.12.22.12.12.12.0 | 2.12-RC1 | 2.12-beta).

Improvements
  • ImageMosaic should work when the images have no CRS information
  • Upgrade Apache POI dependencies
  • Upgrade jasypt dependency
  • Upgrade json-lib dependency to 2.4
  • Upgrade bouncycastle provider to 1.60
Bug Fixes
  •  NullPointerException during WMS request of layer group when caching is enabled
  • GeorectifyCoverage fails to handle paths with spaces
  •  CSS translator does not support mark offset/anchors based on expressions (but SLD does)
  • GeoServerSecuredPage might not redirect to login page in some obscure cases after Wicket upgrade
Security updates

Please update your production instances of GeoServer to receive the latest security updates and fixes.

This release addresses several security vulnerabilities:

  • Prevent arbitrary code execution via Freemarker Template injection
  • XXE vulnerability in GeoTools XML Parser
  • XXE vulnerability in WPS Request builder
  • Various library upgrades (see above) from versions with known CVEs

Thanks to Steve Ikeoka, Kevin Smith, Brad Hards and Nuno Oliveira for providing fixes to these issues.

These fixes are also included in the 2.13.2 release.

If you encounter a security vulnerability in GeoServer, or any other open source software, please take care to report the issue in a responsible fashion.

About GeoServer 2.12 Series

Additional information on the 2.12 series:

Categories: OSGeo Planet

OTB Team: A remote module for Deep learning

OSGeo Planet - Fri, 2018-08-24 14:36
Deep learning has proven to be an extremely powerful tool in many fields, and particularly in image processing: these approaches are currently subject of great interest in the Computer Vision community. However, while a number of typical CV problems can be directly transposed in Remote Sensing (Semantic labeling, Classification problems, …), some intrinsic properties of […]
Categories: OSGeo Planet

Paulo van Breugel: Data exploration in GRASS GIS – boxplots

OSGeo Planet - Fri, 2018-08-24 10:08
I am currently working on some exercises for which I need data about municipalities in the Netherlands. A good place to look for such data is the CBS (Dutch Central Bureau of Statistics). One data layer is vector layers of the dutch municipalities and neighborhoods, which include demographic data. One of the first things I …

Continue reading Data exploration in GRASS GIS – boxplots

Categories: OSGeo Planet

CARTO Inside Blog: COPY'ing with the Python SDK

OSGeo Planet - Thu, 2018-08-23 12:43

Are you a python developer? Have you ever implemented some python code to produce, load or process some data and then wanted to upload it to CARTO? Here is how.

Introducing the CARTO Python SDK’s COPY methods

A few weeks ago Paul posted about our new endpoints in the SQL API to use the PostgreSQL COPY statement for bulk ingestion of data into CARTO.

Although writing a client with requests library can be relatively straightforward, getting it right may be tricky. That’s why we decided to implement what we hope will be a reference client for the SQL API.

This implementation takes care of your COPY’s, and does it in a performant way. It takes care of the data to make sure it is transferred in chunks and uses compression by default, resulting in faster data transfer, in a more pythonic fashion.

Show me the code!

Here are some examples of usage of the python SDK for fast data transfer. Hopefully they will inspire you to accomplish greater endeavors with the SDK.

Working with CSV files

This is the simplest task to accomplish: import or export a CSV file. Okay, you can do it with the Import API. But using python and the CopySQLClient you gain great control over the process. Furthermore, it is several times faster than the traditional imports.

To COPY FROM a CSV file (import), you just need to specify the columns and the format in the COPY query and pass it along with the file to the copyfrom_file_path method:

copy_client = CopySQLClient(auth_client) from_query = """COPY copy_example (the_geom, name, age) FROM stdin WITH (FORMAT csv, HEADER true)""" result = copy_client.copyfrom_file_path( from_query, 'copy_from.csv' )

Similarly, to COPY TO a CSV file (export), you specify a table or a query and the output file and you just get it done:

to_query = """COPY copy_example TO stdout WITH (FORMAT csv, HEADER true)""" copy_client.copyto_file_path(to_query, 'export.csv')

Pro tip: In the COPY TO query you can specify an arbitrary subquery. E.g:

COPY (SELECT * FROM copy_example WHERE cartodb_id <= 1000) TO stdout WITH (FORMAT csv, HEADER true)

Here’s the complete source code of a working example demonstrating the basic usage of the methods above.

Transferring tables

The example above shows how to deal with CSV files stored on the disk.

However, you may want to transfer some data that lives within your script and that does not need to be stored just for the sake of transferring it. Here’s how to pipe the data from a table in an account to another table in another account:

response = copy_src_client.copyto( 'COPY %s TO STDOUT' % TABLE_NAME ) result = copy_dst_client.copyfrom( 'COPY %s FROM STDIN' % TABLE_NAME, response )

As simple as it gets, doesn’t it?

Here’s the complete source of a tool that lets you transfer tables across CARTO accounts. It does work across clouds and on-premise installations. Enjoy it!

Loading data from external services

The most interesting use case, in my opinion, is when you use python for extracting and processing some data, and then you want to transfer it.

As a demonstration of this concepts, you can find here a tool that gets fresh radar information, process it with numpy, and uploads it straight to CARTO. The resulting dataset was then used to produce the cover image of this post.

Use the Source, Luke

You can find here the documentation for the CopySQLClient as well as the reference documentation. That should be your next stop if you want to make the most of this new interface.

Here’s the source code repository. As always, we’re open to contributions via issues and PR’s.

What’s next

We plan to integrate this with CARTOFrames, so that its users can benefit from the speed boost.

We may also extend the interface of CopySQlClient in order to keep the implementation of client apps and integrations as simple as possible while keeping all the advantages of a good usage of the SQL API and underlying services.

Let us know if you have any feedback and suggestions.

Thanks for reading!

Categories: OSGeo Planet

GeoTools Team: GeoTools 18.5 Released

OSGeo Planet - Tue, 2018-08-21 17:52
The GeoTools team is pleased to announce the release of GeoTools 18.5: geotools-18.5-bin.zip geotools-18.5-doc.zip geotools-18.5-userguide.zip geotools-18.5-project.zip maven repository Thanks to everyone who contributed to this release. This release is made in conjunction with GeoServer 2.12.5. This release is a maintenance release providing bug fixes and security updates for production
Categories: OSGeo Planet

GeoSolutions: GeoSolutions presentations from FOSS4G Europe 2018!

OSGeo Planet - Mon, 2018-08-20 13:35

FOSS4G EU 2018

Dear Reader,

we are putting together in this post all presentations that were given by our staff during this year FOSS4G Europe in Guimaraes, Portugal, from in late July.

Here is the complete list. Enjoy!

State of GeoServer 2018 Serving earth observation data with GeoServer: addressing real world requirements Creating Stunning Maps in GeoServer, with SLD, CSS, YSLD and MBStyles Crunching Data In GeoServer: Mastering Rendering Transformations, WPS And SQL Views One GeoNode, many GeoNodes Mapping beyond web mercatorMapStore 2, the story

If you want further information, do not hesitate to contact us.

The GeoSolutions Team,

320x100_eng
Categories: OSGeo Planet
Syndicate content