OSGeo Planet

GIScussions: Esri isn’t evil

OSGeo Planet - Fri, 2017-03-24 16:42

Yup you read it here, an open source advocate said that the leading GI proprietary vendor is not evil. Let me go one step further and say that I think Esri are pretty damn good. Now that is not to say that there are not aspects of their business model and practices that I wouldn’t criticise but more of that in another post.

Come to the Dark Side.

So what has prompted this outburst from an open source advocate? A mail from Suchith Anand to the OSGeo Discuss mailing list entitled “Is it possible for properitery GIS vendor to market thier properitery product as Open ?” (sic).

Hi all,

I have a query. If a properitery GIS vendor starts marketing thier properitery products as Open platform and software then what rights do the organisations and customers have who are mislead buying the  properitery software thinking it is open have ?  The definision of Proprietary software is very clearly defined, so  how can it be possible for any properitery GIS vendor to market their  software knowingly as open platform if it is properitery?

This also greatly affects the business and revenues of true open source software companies .  Who is responsible for any misleading marketing that results in losses to both customers who are mislead to buy the properitery software thinking it is open  and also to other companies who do true open source business who lose out on the business opportunities? Is it right business ethics to do this?

Best wishes,

This has prompted several thoughts which I want to share in a wider format than the OSGeo mailing list (I will post this blog back to the list) partly because the list tends to be a bubble of OSGeo geeks and activists and I want to share these thoughts with less committed users or those who have not yet chosen open source.

I am going to try to deconstruct the propositions of the initial mail (others, particularly Jody Garnett have also expressed contrary views in the mail thread).

a properitery GIS vendor” I am pretty sure that this unnamed vendor is Esri inc and that the mail was prompted by Esri’s Open Vision which would understandably cause a few ruffled feathers and raised eyebrows amongst open source advocates.

The argument seems to be that proprietary software cannot be open, in my opinion that is patently incorrect. We use the word ‘open’ with regard to software and services in numerous different ways e.g. Open Source, Open Standards, Open Data, Open Access and some people even confuse or conflate open with free. The open source community does not have a copyright or any other ownership of the ‘open’ and cannot claim to be the arbiter of who can use the adjective ‘open’ or in what context. As a community we cannot even agree amongst ourselves on the differences between ‘open’ ‘free’ and ‘libre’ (watch this very good presentation on the Free, Open & Libre by María Arias de Reyna).

Furthermore the use of ‘open’ in Open Standards is specifically intended to encourage all (proprietary or open source) to support standards that enable data sharing, as the OGC’s welcome says:

The OGC (Open Geospatial Consortium) is an international not for profit organization committed to making quality open standards for the global geospatial community. These standards are made through a consensus process and are freely available for anyone to use to improve sharing of the world’s geospatial data.

Our members come from government, commercial organizations, NGOs, academic and research organizations.

Open Standards facilitate interoperability and limit ‘vendor lock-in’ which would hardly work without major proprietary vendors supporting the OGC and its standards. ‘Open’ is not the exclusive terminology of the open source community. Whilst we may not like it proprietary software can be described as ‘open’ even if the source code is not freely available.

And let’s be honest about this, apart from a very small number of users who really wants access to source code or has the ability to fix or enhance the code? For many users of open source software the cost of ownership is a major factor and in that case the ‘free’ bit is probably more important than the ‘open’ bit (although the cost of ownership is much more than the license cost).

If a properitery GIS vendor starts marketing thier properitery products as Open platform and software then what rights do the organisations and customers have who are mislead buying the  properitery software thinking it is open have?” Most businesses marketing software (regardless of whether it is open source or proprietary) consider that their potential clients have made a mistake if they eventually succumb to the sales patter of a competitor, we all think we have the ‘best’ offer (specification, quality, value). I doubt that many organisations that purchase Esri software consider themselves to have been mislead by their marketing materials about openness.

Sometimes the open source community (and I include myself in this mea culpa) can position themselves on the side of the angels in a battle between good and evil. That is bullshit! We deliver solutions to customers’ needs that hopefully enable them to harness the power of geography to manage assets, make better decisions and inform stakeholders, the choice between proprietary and open source software is made by informed buyers on the basis of what is best for their organisation in terms of functionality, service, lifetime cost and a whole range of other criteria. Sometimes proprietary solutions are the best answer and sometimes (increasingly so in my opinion) open source represents the most scalable and cost effective solution. The geo market is shifting rapidly from enterprise software implementations to services in which the underlying technology is less visible and less important to the buyer.

Rather than trying to wrestle over the usage of ‘open’ by Esri, open source businesses could learn a fair bit from the effectiveness of Esri’s marketing in building such a successful business that dominates our market. Along the way, whilst making profits (not a dirty word) Esri and in particular Jack Dangermond have done an enormous amount to promote the use of GI, provided employment for thousands of talented people and created the GIS category which has enabled many other business to prosper. They also give the OSGeo a nice big target to aim at!



Categories: OSGeo Planet

gvSIG Team: The 2nd gvSIG Festival is underway!

OSGeo Planet - Fri, 2017-03-24 09:48

After the successful first gvSIG Festival in the last year, the virtual gvSIG conference, a second edition will be held this year, in May 16th and 17th. The novelty of this year is that users can send their proposals about projects done with gvSIG, that will convert this initiative in a more global and open event.

This event is free of charge and completely online, through the webinar service of the gvSIG Association, with the advantage to count with speakers from different countries and where users and developers from any part of the world can hold them.

If you have done any project with gvSIG and you want to present it at the gvSIG Festival, you can send a summmary to the following e-mail address: conference-contact@gvsig.com, explaining the project. The summary will have no more than 300 words, it has to be written in Spanish or English, and you have to indicate the title and the language of the presentation.

Once the program is configured it will be published at the event website, and registrations will be opened.

We expect your proposals!

Filed under: community, english, events, training Tagged: gvSIG Festival
Categories: OSGeo Planet

gvSIG Team: ¡El 2º gvSIG Festival ya está en marcha!

OSGeo Planet - Fri, 2017-03-24 09:44

Tras el éxito del año pasado del primer gvSIG Festival, las jornadas virtuales sobre gvSIG, se va a celebrar este año una segunda edición, los próximos días 16 y 17 de mayo, en las que como novedad de este año los usuarios podrán enviar sus propuestas de proyectos sobre la aplicación, lo que convierte esta iniciativa en un evento todavía más global y abierto.

Este evento es gratuito y completamente online, a través del servicio de webinar en la Asociación gvSIG, con la ventaja de poder contar con ponentes de diferentes países, y donde las ponencias pueden ser seguidas por usuarios de cualquier parte del mundo.

Si has realizado algún proyecto con gvSIG y quieres presentarlo en el gvSIG Festival, puedes enviar un resumen explicando en qué consiste a la dirección de correo conference-contact@gvsig.com. El resumen deberá ser en español o inglés, de un máximo de 300 palabras, y en él se deberá indicar el título de la ponencia y el idioma en el que se daría.

Una vez configurado el programa se publicará en la web del evento y se abrirán las inscripciones para cada presentación.

¡Esperamos vuestras propuestas!

Filed under: community, events, spanish, training Tagged: gvSIG Festival
Categories: OSGeo Planet

Tom Kralidis: pygeometa: new release, hello YAML

OSGeo Planet - Thu, 2017-03-23 20:13
Metadata should be automagic and low barrier. pygeometa is a handy little metadata generator tool which is flexible, extensible, and composable.  Command line, or via the API, users can generate config files, or pass plain old Python dicts, ConfigParser objects, etc. We’ve just released 0.2.0 which supports WMO Core Metadata Profile output, as well as […]
Categories: OSGeo Planet

gvSIG Team: Aprende a trabajar con Modelos Digitales del Terreno y geoprocesamiento ráster con este vídeo-tutorial

OSGeo Planet - Thu, 2017-03-23 09:19

Complementando el post en el que os enseñamos los secretos del geoprocesamiento vectorial, hoy veremos como podemos aplicar algunos de los algoritmos disponibles en gvSIG Desktop a las capas ráster en general, y a los Modelos Digitales del Terreno en particular.

De los más de 350 geoprocesos disponibles en gvSIG, un buen número de ellos son aplicables a capas ráster, permitiéndonos hacer cálculos de todo tipo y que son especialmente útiles para algunas disciplinas científicas como la hidrología.

Mediante una serie de ejercicios prácticos, que podréis replicar en vuestra casa, este vídeo-tutorial os mostrará en pocos minutos como realizar geoprocesamiento ráster. Seguid leyendo….

Filed under: gvSIG Desktop, spanish Tagged: geoprocesamiento, hidrología, MDT, Modelo Digital del Terreno, raster
Categories: OSGeo Planet

Jackie Ng: React-ing to the need for a modern MapGuide viewer (Part 15): Play with it on docker

OSGeo Planet - Wed, 2017-03-22 16:22
Today, I found a very interesting website from the tech grapevine:


What is this site? It is an interactive docker playground. If you've ever used sites like JSFiddle to try out snippets of JS/HTML/CSS, this is basically the docker equivalent to try out Docker environments.

With PWD, I now have a dead simple way for anyone who wants to try out this viewer to spin up a demo MapGuide instance on PWD for themselves to check out the viewer.

Once you've proved to the site that you're are indeed a human and not a robot, you will enter the PWD console. From here, click + ADD NEW INSTANCE to start a new shell.

Then run the following commands to build the demo docker image and spin up the container

git clone https://github.com/jumpinjackie/mapguide-react-layout
cd mapguide-react-layout

After a few minutes, you should see a port number appear beside the IP address

This is a link to the default Apache httpd page that is confirmation that the demo container is serving out web content to the outside world.

Now simply append /mapguide/index.php to that URL to access the demo landing page for this viewer. Pick any template on the list to load the viewer using that template.

You now have a live demo MapGuide Server with mapguide-react-layout (and the Sheboygan dataset) preloaded for you to play with to your heart's content for the next 4 hours, after which PWD will terminate your session and all the docker images/containers/etc that you created with it.

This was just one use case that I thought up in 5 minutes after discovering this awesome site! I'm sure there's plenty of other creative uses for such a site like this.

Many thanks to brucepc for his MGOS 3.1 docker image from which the demo image is based from.
Categories: OSGeo Planet

Jackie Ng: gRPC is very interesting

OSGeo Planet - Wed, 2017-03-22 16:19
MapGuide in its current form is a whole bucket of assorted libraries and technologies:
  • We use FDO for spatial data access
  • We use ACE (Adaptive Communication Environment) for:
    • Basic multi-threading primitives like mutexes, threads, etc
    • TCP/IP communication between the Web Tier and the Server Tier
    • Implementing a custom RPC layer on top of TCP/IP sockets. All of the service layer methods you use in the MapGuide API? They're all basically RPC calls sent over TCP/IP for the MapGuide Server to invoke its server-side eqvivalent. Most of the other classes that you pass into these service methods are essentially messages that are serialized/deserialized through the TCP/IP sockets. When you think about it, the MapGuide Web API is merely an RPC client for the MapGuide Server, which itself is an RPC server that does the actual work
  • We use Berkeley DBXML for the storage of all our XML-based resources
    • We have an object-oriented subset of these resource types (Feature Sources, Layer Definitions, Map Definitions, Symbol Definitions) in the MdfModel library with XML serialization/parsing code in the MdfParser library
    • Our Rendering and Stylization Engine work off of these MdfModel classes to render the maps that you see on your viewer
  • We use xerces for XML reading/writing XML in and out of DBXML
  • We use a custom modified (and somewhat ancient) version of SWIG to generate wrappers for our RPC client so that you can talk to the MapGuide Server in:
    • .net
    • Java
    • PHP
So why do I mention all of this?

I mention this, because I've recently been checking out gRPC, a cross-platform, cross-language RPC framework from Google.

And from what I've seen so far, gRPC could easily replace and simplify most of the technology stack we're currently using for MapGuide:
  • ACE? gRPC is the RPC framework! The only reason we'd keep ACE around would be for multi-threading facilities, but the C++ standard library at this point would be adequate enough to replace that as well
  • DBXML/MdfModel/xerces? gRPC is driven by Google Protocol Buffers.
    • Protobuf messages are strongly typed classes that serialize/deserialize into compact binary streams and is more efficient and faster than slinging around XML. Ever bemoan the fact you have to currently work with XML to manipulate maps/layers/etc? In .net you are reprieved if you use the Maestro API (where we provide strongly-typed classes for all the resource XML types), but for the other languages you have to figure out how to use the XML APIs/services provided by Java/PHP to work with the XML blobs that the MapGuide API gives and expects. With protobuf, you have none of these problems.
    • Protobuf messages can evolve in a backward-compatible manner
    • Because protobuf messages are already strongly-typed classes, it makes MdfModel/MdfParser redundant if you get the Rendering/Stylization engine to work against protobuf messages for maps/layers/symbols/styles/etc
    • If we ever wanted to add support for Mapbox Vector Tiles (which seems to be the de-facto vector tile format), well the spec is protobuf-based so ...
    • Protobuf would mean we no longer deal in XML, so we don't need Xerces for reading/writing XML and DBXML as the storage database (and all its cryptic error messages that can bubble up from the Resource Service APIs) can be replaced with something simpler. We may not even need a database at this point. Dumping protobuf messages to a structured file system could probably be a simpler solution
  • SWIG? gRPC and protobuf can already generate service stubs and protobuf message classes in the languages we currently target:
    • .net
    • Java
    • PHP
    • And if we wanted, we can also instantly generate a gRPC-based MapGuide API for:
      • node.js
      • Ruby
      • Python
      • C++
      • Android Java
      • Objective-C
      • Go
    • The best thing about this? All of this generated code is portable in their respective platforms and doesn't involve native code interop through "flattened" interfaces of C code wrapping the original C++ code, which is what SWIG ultimately does for any language we want to generate wrapper bindings out of. If it does involve native code interop, it's a concern that is taken care of by the respective gRPC/protobuf implementation for that language.
  • Combine a gRPC-based MapGuide Server with grpc-gateway and we'd have an instant REST API to easily build a client-side map viewer out of
  • gRPC works at a scale that is way beyond what we can currently achieve with MapGuide currently. After all, this is what Google uses themselves for building their various services
If what I said above doesn't make much sense, consider a practical example.

Say we had our Feature Service (which as a user of the MapGuide API, you should be familiar with) as a gRPC service Definition

// Message definitions for the request/response types below are omitted for brevity but basically every request and
// response type mentioned below will have eqvivalent protobuf message classes automatically generated along with
// the service

// Provides an abstraction layer for the storage and retrieval of feature data in a technology-independent way.
// The API lets you determine what storage technologies are available and what capabilities they have. Access
// to the storage technology is modeled as a connection. For example, you can connect to a file and do simple
// insertions or connect to a relational database and do transaction-based operations.
service FeatureService {
// Creates or updates a feature schema within the specified feature source.
// For this method to actually delete any schema elements, the matching elements
// in the input schema must be marked for deletion
rpc ApplySchema (ApplySchemaRequest) returns (BasicResponse);
rpc BeginTransaction (BeginTransactionRequest) returns (BeginTransactionResponse);
// Creates a feature source in the repository identified by the specified resource
// identifier, using the given feature source parameters.
rpc CreateFeatureSource (CreateFeatureSourceRequest) returns (BasicResponse);
rpc DeleteFeatures (DeleteFeaturesRequest) returns (DeleteFeaturesResponse);
// Gets the definitions of one or more schemas contained in the feature source for particular classes.
// If the specified schema name or a class name does not exist, this method will throw an exception.
rpc DescribeSchema (DescribeSchemaRequest) returns (DescribeSchemaResponse);
// This method enumerates all the providers and if they are FDO enabled for the specified provider and partial connection string.
rpc EnumerateDataStores (EnumerateDataStoresRequest) returns (EnumerateDataStoresResponse);
// Executes SQL statements NOT including SELECT statements.
rpc ExecuteSqlNonQuery (ExecuteSqlNonQueryRequest) returns (ExecuteSqlNonQueryResponse);
// Executes the SQL SELECT statement on the specified feature source.
rpc ExecuteSqlQuery (ExecuteSqlQueryRequest) returns (stream DataRecord);
// Gets the capabilities of an FDO Provider
rpc GetCapabilities (GetCapabilitiesRequest) returns (GetCapabilitiesResponse);
// Gets the class definition for the specified class
rpc GetClassDefinition (GetClassDefinitionRequest) returns (GetClassDefinitionResponse);
// Gets a list of the names of all classes available within a specified schema
rpc GetClasses (GetClassesRequest) returns (GetClassesResponse);
// Gets a set of connection values that are used to make connections to an FDO provider that permits multiple connections.
rpc GetConnectionPropertyValues (GetConnectionPropertyValuesRequest) returns (GetConnectionPropertyValuesResponse);
// Gets a list of the available FDO providers together with other information such as the names of the connection properties for each provider
rpc GetFeatureProviders (GetFeatureProvidersRequest) returns (GetFeatureProvidersResponse);
// Gets the locked features.
rpc GetLockedFeatures (GetLockedFeaturesRequest) returns (stream FeatureRecord);
// Gets all available long transactions for the provider
rpc GetLongTransactions (GetLongTransactionsRequest) returns (GetLongTransactionsResponse);
// This method returns all of the logical to physical schema mappings for the specified provider and partial connection string
rpc GetSchemaMapping (GetSchemaMappingRequest) returns (GetSchemaMappingResponse);
// Gets a list of the names of all of the schemas available in the feature source
rpc GetSchemas (GetSchemasRequest) returns (GetSchemasResponse);
// Gets all of the spatial contexts available in the feature source
rpc GetSpatialContexts (GetSpatialContextsRequest) returns (GetSpatialContextsResponse);
// Inserts a new feature into the specified feature class of the specified Feature Source
rpc InsertFeatures (InsertFeaturesRequest) returns (stream FeatureRecord);
// Selects groups of features from a feature source and applies filters to each of the groups according to the criteria set in the aggregate query option supplied
rpc SelectAggregate (SelectAggregateRequest) returns (stream DataRecord);
// Selects features from a feature source according to the criteria set in the query options provided
rpc SelectFeatures (SelectFeaturesRequest) returns (stream FeatureRecord);
// Set the active long transaction name for a feature source
rpc SetLongTransaction (SetLongTransactionRequest) returns (BasicResponse);
// Connects to the Feature Provider specified in the connection string
rpc TestConnection (TestConnectionRequest) returns (TestConnectionResponse);
// Executes commands contained in the given command set
rpc UpdateFeatures (UpdateFeaturesRequest) returns (UpdateFeaturesResponse);
// Updates all features that match the given filter with the specified property values
rpc UpdateMatchingFeatures (UpdateMatchingFeaturesRequest) returns (UpdateMatchingFeaturesResponse);

Running this service definition through the protoc compiler with grpc plugin gives us:
  • Auto-generated (and strongly-typed) protobuf classes for all the messages. ie: The request and response types for this service
  • An auto-generated FeatureService gRPC client ready to use in the language of our choice
  • An auto-generated gRPC server stub for FeatureService in the language of our choice ready for us to "fill in the blanks". For practical purposes, we'd generate this part in C++ and fill in the blanks by mapping the various service operations to their respective FDO APIs and its return values to our gRPC responses.
And at this point, we'd just need a simple C++ console program that bootstraps gRPC/FDO, registers our gRPC service implementation, start the gRPC server on a particular port and we'd have a functional Feature Service implementation in gRPC. Our auto-generated Feature Service client can connect to this host and port to immediately start talking to it.

The only real work is the "filling in the blanks" on the server part. Everything else is taken care of for us.

Extrapolate this to the rest of our services (Resource, Rendering, etc) and we basically have a gRPC-based MapGuide Server.

Also filling in the blanks is a conceptually simple exercise as well:
  • Feature Service - Pass down the APIs in FDO.
  • Rendering Service - Setup up FDO queries based on map/layers visible and pass query results to the Rendering/Stylization engine.
  • Resource Service - Read/write protobuf resources to some kind of persistent storage. It doesn't have to be something complex like DBXML, it can be as simple as a file system (that's what mg-desktop does for its resource service implementation btw)
  • Tile Service - It's just like the rendering service, but you're asking the Rendering/Stylization engine to render tile-sized content.
  • KML Service - Just like rendering service, but you're asking the Rendering/Stylization engine to render KML documents instead of images.
  • Drawing Service - Do we still care about DWF support? Well if we have to support this, it's just passing down to the APIs in DWF Toolkit.
  • Mapping Service - It's a mish-mash of tapping into the Rendering/Stylization engine and/or the DWF Toolkit.
  • Profiling Service - Just tap into whatever tracing/instrumentation APIs provided by gRPC.
Now because gRPC is cross-language, nothing says we have to use C++ for all the service implementations, it's just that most of the libraries and APIs we'd be mapping into are already in C++, so in practical terms we'd stick to the same language as well.

Front this with grpc-gateway, and we basically have our RESTful mapagent to build a map viewer against.

There's still a few unknowns:
  • How do we model file uploads/downloads?
  • Can server-side service implementations call other services?
Google's vast set of gPRC definitions for their various web services can answer the first part. The other part, will need more playing around.

The thought of a gRPC-based MapGuide Server is very exciting!
Categories: OSGeo Planet

gvSIG Team: gvSIG nominado en 3 categorías de los “Open Awards”. Abierta votación popular.

OSGeo Planet - Tue, 2017-03-21 15:45

El proyecto gvSIG vuelve a ver reconocida su propuesta de desarrollo de soluciones de geomática libre con la nominación a 3 categorías de los premios denominados “Open Awards”.

Tal y como se indica en la web de los premios, estos tienen como objetivo reconocer públicamente a empresas, administraciones, personalidades y comunidades que crean, apoyan y fomentan grandes soluciones con tecnologías Open Source y Software Libre.

Los Open Awards reconocen y premian los proyectos e iniciativas de código abierto que más han destacado durante el último año, impulsan la comunicación y la notoriedad pública de las empresas, proyectos y administraciones participantes en los premios y valoran el trabajo realizado por todos ellos.

Las 3 categorías en las que participa gvSIG reflejan en cierto modo la mirada amplia del proyecto, ya que se opta al “Mejor proveedor de servicios/soluciones” donde la Asociación gvSIG está demostrando que pueden ponerse en marcha nuevos modelos económicos de producción de software desde perspectivas colaborativas; a la “Plataforma/proyecto más innovador” una categoría que se centra en la parte técnico-científica del proyecto y donde gvSIG Suite se constituye como la plataforma integral para solventar las necesidades “geo” de cualquier organización, con aplicaciones de escritorio, móviles y web siguiendo la filosofía de las Infraestructuras de Datos Espaciales; por último la categoría donde más ilusión nos hace participar y que en realidad es un reconocimiento a todos vosotros, los que día a día, desde vuestro rincón apoyáis y ayudáis a gvSIG, la nominación a “Mejor comunidad tecnológica”.

En estos premios hay una primera fase de votaciones públicas de la que saldrá un “top cinco”. A partir de ahí un jurado determinará los ganadores de cada categoría.

¿Qué pasos hay que seguir para votad?

  1. Votad en las categorías que consideres (desde hoy hasta el 30 de abril).
  2. Confirmad voto por email.

Como aliciente adicional, por votar se tiene acceso exclusivo al eBook “OpenExpo Tendencias Open Source y Software Libre 2017”.

Los enlaces directos a las categorías donde está gvSIG nominado son:

Desde ya os agradecemos a todos vosotros vuestro apoyo y tened seguro que nosotros también os votaremos como “Mejor comunidad tecnológica”.

Categories: OSGeo Planet

GeoSolutions: GeoServer Code Sprint needs you

OSGeo Planet - Tue, 2017-03-21 12:22


Dear Reader,

everything is ready in GeoSolutions for next week's GeoServer code sprint which will take place in our offices during the week of March 27th.

[caption id="attachment_3380" align="aligncenter" width="531"]Sprint 2017 Sprint 2017[/caption]

The main focus will be on refactoring GeoServer's REST API towards a more modern approach (se this page for some insights). A number of GeoServer developers from various organizations will gather from all over world for this work and your support in funding this initiative would help us out with the expenses, therefore, we are asking help to all our readers. Sponsorship opportunities are available for you to contribute on the OSGeo wiki.

Happy sprinting to everybody! The GeoSolutions team,
Categories: OSGeo Planet

gvSIG Team: Aprende los secretos del geoprocesamiento vectorial en gvSIG con este vídeo-tutorial

OSGeo Planet - Tue, 2017-03-21 09:01

En gvSIG Desktop disponemos de más de 350 geoprocesos, esto sin contar con plugins como el recientemente anunciado de Jgrass. Un buen porcentaje de esos geoprocesos se aplican sobre capas vectoriales, desde los más comunes -como el área de influencia, cortar, unir,…- a otros más específicos y menos conocidos.
Hoy os presentamos un vídeo-tutorial en el que en pocos minutos aprenderéis el funcionamiento de los geoprocesos de gvSIG, mediante una serie de ejercicios prácticos que nos permitan comprender la sencillez con la que podemos utilizar los algoritmos disponibles en la aplicación.
En la parte final del vídeo-tutorial podréis aprender a manejar el modelador de geoprocesos; una herramienta muy útil y no muy conocida por los usuarios de gvSIG.
Seguid leyendo…

Filed under: gvSIG Desktop, spanish Tagged: geoprocesamiento, geoprocesos, modelador
Categories: OSGeo Planet

GeoSolutions: New release of MapStore 2 with theming support

OSGeo Planet - Mon, 2017-03-20 11:03


Dear Reader,

we are pleased to announce a new release of MapStore 2, our flagship Open Source webgis product, which we have called 2017.02.00. The full list of changes for this release can be found here, but let us now concentrate on the latest most interesting additions.

Advanced Theming The main Feature of this release is the possibility to have different themes, as shown in the Gallery below. [gallery type="slideshow" ids="3319,3320,3324,3325,3326,3321,3327,3328,3329,3330"]

MapStore 2 was conceived with the goal to be highly customizable, therefore we have worked hard on the look and feel from the beginning to create a product that would easily adapt to predefined graphical guidelines as well as a framework which could be easy to integrate with 3rd party applications.

With this release the goal has been achieved. We have refactored the original theme simplifying greatly the steps to create new themes and switch between them. You can try to switch it live from the home page, there is a specific combo with some predefined styles.

On the technical side, we have refactored MapStore 2 theme support using less, hence now creating your own theme to match your company's visual design guidelines is very simple. We are developing an example that allows you to customize your theme directly from the web page, you can see it here below or test it here.

[gallery type="slideshow" ids="3333,3334"] Balloon Tutorial The balloon tutorial is now ready also with html support. You can try it live by clicking on "Tutorial" in the map's burger menu, as an instance in this map. [gallery type="slideshow" ids="3336,3337,3338,3339,3340"]   Notes for Developers

This release has a number of changes that are crucial to know for the developers since they break compatibility with older versions. Here, you can find the details of what we updated and how to migrate your application.

We strongly believe that these changes will speed up the development and improve the quality and the readability of the code (particularly redux-observable). If you will find yourself struggling with these changes, reach out for us on the developer mailing list and we will help you out.

We also looked around for a tool to produce developers' docs that would satisfy our needs in the longer term.  We found in docma a great tool as it allows us to provide both generic guides as well as to document our components, plugins and JavaScript API inline using jsDoc + Markdown. You can find the current version of the developer documentation here. Twitter Account

MapStore 2 has now its own twitter account which it is using to let us know how it feels as well as to share useful information and insights.

What we are working on for the next release

The main focus for the next release is the implementation of a JavaScript API to allow you to include MapStore 2 in your application or web-site and interact with it in more advanced ways than a simple IFRAME. We are also going to focus on the following items:

  • Improve developer's documentation
  • Improve the management of Maps, in order to allow users to manage them also from the map itself
  • Better interaction with WFS

In the longer term, we have a number of features and functionalities in our plans like editing, advanced templating, styling, OAUTH 2.0, and more…

So, Stay tuned and happy webmapping!

If you are interested in learning about how we can help you achieving your goals with open source products like GeoServerMapstore, GeoNode and GeoNetwork through our Enterprise Support Services and GeoServer Deployment Warranty offerings, feel free to contact us!

The GeoSolutions team,
Categories: OSGeo Planet

gvSIG Team: Disponible vídeo-tutorial para aprender Geoestadística con gvSIG

OSGeo Planet - Wed, 2017-03-15 09:06

El paquete estadístico R es uno de los más flexibles, potentes y profesionales que existen actualmente para realizar tareas estadísticas de todo tipo, desde las más elementales, hasta las más avanzadas. Y, lo más importante, es software libre.

Desde sus últimas versiones, gvSIG Desktop ha incluido plugins para integrar R, abriendo así la posibilidad a realizar todo tipo de análisis geoestadísticos.

Geólogos, biólogos, ecólogos, agrónomos, ingenieros, meteorólogos, sociólogos…por nombrar sólo unos pocos profesionales, requieren del estudio de información estadístico de información georreferenciada.

Desde la Asociación gvSIG os presentamos un vídeo-tutorial que os permitirá introduciros en el funcionamiento de la dupla gvSIG-R.

Si hemos despertado tú interés, sigue leyendo…

Filed under: gvSIG Desktop, spanish Tagged: Geoestadística, r
Categories: OSGeo Planet

GeoServer Team: GeoServer 2.11-RC1 Released

OSGeo Planet - Tue, 2017-03-14 07:23

We are happy to announce the release of GeoServer 2.11-RC1. Downloads are available (zipwardmg and exe) along with docs and extensions.

This is a release candidate of GeoServer not intended for production use. This release is made in conjunction with GeoTools 16-RC1 and GeoWebCache 1.11-RC1.

Thanks to everyone who provided feedback, bug reports and fixes. Here are some of the changes included in 2.11-RC1:

  • Incompatibilities with GeoFence and Control-flow have been resolved
  • Empty WFS Transactions (which perform no changes) no indicating everything has changed
  • Improvements to WFS GetFeature support for 3D BBOX requests
  • We have one known regression with the windows service installer (memory setting is incorrect)
  • Please additional details see the release notes (2.11-RC12.11-beta )
Release Candidate Testing

The 2.11 release is expected in March, this release candidate is a “dry run” where we confirm new functionality is working as expected and double check the packaging and release process.

Please note that GeoServer 2.9 has reached its end-0f-life. If your organization has not yet upgraded please give us hand by evaluating 2.11-RC1 and providing feedback and your experiences for the development team. This is a win/win situation where your participation can both assist the GeoServer team and reduce your risk when upgrading.

Corrected default AnchorPoint for LabelPlacement

An issue with SLD 1.0 rendering has been fixed – when a LabelPlacement did not include a AnchorPoint we were using the wrong default!

  • BEFORE: default anchor point was X=0.5 and Y=0.5 – which is at the middle height and middle length of the label text.
  • AFTER: default anchor point was X=0.0 and Y=0.5 – which is at the middle height of the lefthand
    side of the label text.

This is a long standing issue that was only just noticed in February. If you need to “restore” the incorrect behaviour please startup with -Dorg.geotools.renderer.style.legacyAnchorPoint=true system property.

Startup Performance

With extensive improvements to startup performance and OGC requests for large installations we are looking forward to feedback from your experience testing.

About GeoServer 2.11

GeoServer 2.11 is scheduled for March 2017 release. This puts GeoServer back on our six month “time boxed” release schedule.

  • OAuth2 for GeoServer (GeoSolutions)
  • YSLD has graduated and is now available for download as a supported extension
  • Vector tiles has graduate and is now available for download as an extension
  • The rendering engine continues to improve with underlying labels now available as a vendor option
  • A new “opaque container” layer group mode can be used to publish a basemap while completely restricting access to the individual layers.
  • Layer group security restrictions are now available
  • Latest in performance optimizations in GeoServer (GeoSolutions)
  • Improved lookup of EPSG codes allows GeoServer to automatically match EPSG codes making shapefiles easier to import into a database (or publish individually).
Categories: OSGeo Planet

Even Rouault: Dealing with huge vector GeoPackage databases in GDAL/OGR

OSGeo Planet - Sat, 2017-03-11 22:25
Recently, I've fixed a bug in the OGR OpenFileGDB driver, the driver made from the reverse engineering the ESRI FileGeoDatabase format, so as to be able to read tables whose section that enumerates and describes fields is located beyond the first 4 GB of the file. This table from the 2016 TIGER database is indeed featuring all linear edges of the USA and is 15 GB large (feature and spatial indexes included), with 85 million features.

Some time before, I had to deal with a smaller database - 1.7 GB as GeoPackage - with 5.4 million polygons (bounding box) from the cadastre of an Italian province. One issue I noticed is that when you want to get the summary of the layer, with ogrinfo -al -so the.gpkg, it was very slow. The reason is that this summary includes the feature count, and there's no way to get this metadata quickly, apart from running the "SELECT COUNT(*) FROM the_table" request, which causes a full scan of the table. For small databases, this runs fast, but when going into the gigabyte realm, this can take several dozains of seconds. But getting the spatial extent of the layer, which is one of the other information displayed by the summary mode of ogrinfo, is fast since the gpkg_contents "system" table of a GeoPackage database includes the bounding box of the table. So my idea was to extend the definition of the gpkg_contents table to add a new column, ogr_feature_count, to store the feature count. I went to implement that, and it worked fine. The synchronization of the value of ogr_feature_count after edits can be done with 2 SQLite triggers, on row insertion and deletion, and that  works with implementations that are not aware of the existence of this new column. Like older OGR versions. Unfortunately it appears that at least one other implementation completely rejected such databases. There is some inconsistency in the GeoPackage specification if additional columns are accepted or not in system tables. From the /base/core/contents/data/table_def test case, "Column order, check constraint and trigger definitions, and other column definitions in the returned sql are irrelevant.", it would seem that additional columns should still be considered as a valid GeoPackage. Anyway, that's only the theory and we don't want to break interoperability for just a nice-to-have feature... So I went to change the design a bit and created a new table, gpkg_ogr_contents, with a table_name and feature_count columns. I'm aware that I should not borrow the gpkg_ prefix, but I felt it was safer to do so since other implementations will probably ignore any unknown gpkg_ prefixed table. And the addition of the ogr_ prefix makes collisions with future extension of the GeoPackage specification unlikely. The content of this table is also maintained in synchronization with the data table thanks to two triggers, and this makes the other software that rejected my first attempt happy. Problem solved.

Let's come back to our 13 GB FileGeoDatabase. My first attempt to convert is to GeoPackage with ogr2ogr resulted in converting the features in about half an hour, but once this 100% stage reached, the finalization, which includes building the spatial index took ages. So long, that after a whole night it wasn't yet finished and seriously making the computer non responsive due to massive I/O activity. In the GeoPackage driver, the spatial index is indeed created after feature insertion, so that the feature table and spatial index tables are well separated in the file, and from previous experiment with the Spatialite driver, it proved to be the right strategy. Populating the SQLite R-Tree is done with a simple statement: INSERT INTO my_rtree SELECT fid, ST_MinX(geom), ST_MaxX(geom), ST_MinY(geom), ST_MaxY(geom) FROM the_table. Analyzing what happens in the SQLite code is not easy when you are not familiar with that code base, but my intuition is that there was constant back and forth between the geometry data area and the RTree area in the file, making the SQLite page cache inefficient. So I decided to experiment with a more progressive approach. Let's iterate over the feature table and collect the fid, minx, max, miny, maxy by chunks of 100 000 rows, and the insert those 100 000 bounding boxes into the R-Tree, and loop again unil we have completely read the feature table. With such a strategy, the spatial index can now be built in 4h30. The resulting GeoPackage file weights 31.6 GB, so twice at large than the FileGeoDatabase. One of the reasons for the difference must be due to geometries in FileGeoDatabase being compressed (quantization for coordinate precision, delta encoding and use of variable integer) whereas GeoPackage uses a uncompressed SQLite BLOB based on OGC WKB.
My first attempt at opening it in QGIS resulted in the UI to be frozen, probably for hours. The reason is that QGIS always issues a spatial filter, even when requesting on a area of interest that is at least as large as the extent of the layer, where there is no performance gain to expect from using it. So the first optimization was in the OGR GeoPackage to detect that situation and to not translate the OGR spatial filter as SQLite R-Tree filter. QGIS could now open the database and progressively displays the features. Unfortunately when zooming in, the UI became frozen again. When applying a spatial filter, the GeoPackage driver created a SQL request like the following one:
SELECT * FROM the_table WHERE fid IN        (SELECT id FROM the_rtree WHERE         xmin <= bbox_xmax AND xmax >= bbox_xmin AND        ymin <= bboy_ymay AND ymay >= bboy_ymin)
It turns out that the sub-select (the one that fetches the feature ID from the spatial index) is apparently entirely run before the outer select (the one that returns geometry and attributes) starts being evaluated. This way of expressing the spatial filter came from the Spatialite driver (since GeoPackage and Spatialite use the exact same mechanisms for spatial indexing), itself based on examples from an old Spatialite tutorial. For not too big databases, this runs well. After some experiment, it turns out that doing a JOIN between the feature table and the RTree virtual table makes it possible to have a non blocking request:
SELECT * FROM the_table t JOIN the_rtree r ON t.fid = r.idWHERE r.xmin <= bbox_xmax AND r.xmax >= bbox_xmin AND      r.ymin <= bboy_ymax AND r.ymax >= bboy_ymin
Now QGIS is completely responsive, although I find that even on high zoom levels, the performance is somehow disappointing, ie features appear rather slowly. There seems to be some threshold effect on the size of the database, since the performance is rather good on the Italian province cadastral use case.

Another experiment showed that increasing the SQLite page size from 1024 bytes (the default in SQLite 3.11 or earlier) to 4096 bytes (the default since SQLite 3.12) decreases the database size to 28.8 GB. This new page size of 4096 bytes is now used by default by the OGR SQLite and GPKG drivers (unless OGR_SQLITE_PRAGMA=page_size=xxxx is specified as a configuration option).

I also discovered that increasing the SQLite page cache from its 2 MB default to 2 GB (with --config OGR_SQLITE_CACHE 2000) significantly improved the time to build the spatial index, decreasing the total conversion time from 4h30 to 2h10. 2GB is just a value selected at random. It might be too large or perhaps a larger value would help further.

All improvements mentionned above (faster spatial index creation, better use of spatial index and change of default page size) are now in GDAL trunk, and will be available in the upcoming 2.2.0 release.
Categories: OSGeo Planet

gvSIG Team: 3rd Catedra gvSIG Contest

OSGeo Planet - Thu, 2017-03-09 18:34

The aim of the Cátedra gvSIG is to create a meeting point for users interested in free space technologies. In order to foment an environment of shared knowledge and participating in the dissemination of free geomatics, the chair organizes this international contest to encourage all gvSIG users and free Geographic Information Systems users to share and give visibility to their work.

Students and graduates in high school, professional training and university, as well as university professors and researchers from all countries can participate in this contest. To enter to the competition you must meet the following requirements: Works must be done with free Geographic Information Systems and the subject of the work may address any area of knowledge. Works may have been made in 2016 or later, the papers may be presented collectively and individually and jobs may be sent in Spanish, Valencian or English.

In the event the work is based on a new development done through free and open source GIS geospatial technologies, these papers must be subjected to GNU / GPL v3 license. Among the selected works a prize of 500 euros will be awarded for each of the following categories:

  • Work produced by students of highschool or professional training.

  • Final University’s Project (Bachelor, Degree or Master).

  • Doctoral thesis or research paper.

Submissions should be sent to gvsig@umh.es and press@gvsig.com no later than November 1, 2017. Selected documents will be published in the repository of the Cátedra gvSIG UMH. The jury will evaluate the methodology, clarity and innovative nature of the work, assessing as well the relevance and applicability of the research.

Winners will be announced in the next International gvSIG Conference.

Filed under: english, events, press office Tagged: awards, contest, open source
Categories: OSGeo Planet

GeoServer Team: REST API Code Sprint Prep

OSGeo Planet - Thu, 2017-03-09 15:34

In our previous blog post we highlighted the GeoServer Code Sprint 2017 taking place at the of this month. We are all looking forward to GeoSolutions hosting us in beautiful Tuscany and have lots of work to do.

One of the secrets (and this comes as no surprise) to having a successful code sprint is being prepared. With this year’s REST API migration from restlet to spring model-view-controller we want to have all technical decisions made, and examples for the developers to work from, prior to any boots hitting the ground in Italy.

But before we get into the details …

Code Sprint Sponsors

We would like to thank our sprint sponsors – we are honoured that so many organizations have stepped up world wide to fund this activity.

Gaia3D is a professional software company in the field of geospatial information and Earth science technology. We would like to thank Gaia3D for their gold sponsorship.


Insurance Australia Group (IAD) is our second gold sponsor. This is a great example of open source being used, and supported, by an engineering team. Thanks to Hugh Saalmans and the Location Engineering team at IAD for your support.


Boundless is once again sponsoring the GeoServer team. Boundless provides a commercially supported open source GIS platform for desktop, server, mobile and cloud. Thanks to Quinn Scripter and the Boundless suite team for their gold sponsorship.



How 2 Map is pleased to support this year’s event with a bronze sponsorship.


I am overjoyed FOSSGIS (German local OSGeo chapter) is supporting us with a bronze sponsorship. This sponsorship means a lot to us as the local chapter program focuses on users and developers; taking the time to support our project directly is a kind gesture.



Sponsorship Still Needed

While we have a couple of verbal commitments to sponsor – we are still $1500 USD off the pace. If your organization has some capacity to financially support this activity we would dearly love your support.

This is an official OSGeo activity; any excess money is returned to the foundation to help the next open source sprint.  OSGeo sponsorship is cumulative. Check their website for details on how your helping out the geoserver team can be further recognized.

For sponsorship details visit the wiki page (or contact Jody Garnett for assistance).

Update: Since this post was published we are happy to announce new sponsor(s).

Thanks to Caroline Chanlon and the team at Atol Conseils et Développements for bronze sponsorship.


Update: Thanks to David Ghedini (acugis.com) and others donating smaller amounts via the OSGeo paypal button.

Getting Ready for REST

In this week’s GeoServer meeting we had a chance to sit down and plan out the steps needed to get ready.

The majority of prep will go into performing the restlet to spring mvc migration for a sample REST API end point to produce a “code example” for developers to follow. We have selected the rest/styles endpoint as one of the easier examples:

  1. Preflight check: Before we start we want to have a good baseline of the current REST API responses. We would like to double check that each endpoint has a JUnit test case that checks the response against a reference file. Most of our tests just count the number of elements, or drill into the content to look for a specific value. The goal is to use these reference files as a quick “regression test” when performing the migration.
  2. Migrate rest/styles from StyleResource (restlet) to StyleController (spring): This should be a bit of fun, part of why spring model-view-controller was selected. Our goal is to have one Controller per end-point, and configure the controller using annotations directly in the Java file. This ends up being quite readable with variable names being taken directly out of the URL path. It is also easier to follow since you do not have to keep switching between XML and Java files to figure out what is going on.  It is important that the example is “picture perfect” as it will be used as a template by the developers over the course of the sprint, and will be an example of the level of quality we expect during the activity.
  3. Create StyleInfo bindings (using XStream for xml and json generation): The above method returns a StyleInfo data structure, our current restlet solutions publishes each “resource” using the XStream library. We think we can adapt our XStream work for use in spring model-view-controller by configuring a binding for StyleInfo and implementing in using XStream.  This approach is the key reason we are confident in this migration being a success; existing clients that depend on exactly the same output from GeoServer – should get exactly the same output.
  4. StyleController path management: There is some work to configure each controller, while we have the option of doing some custom logic inside each controller we would like to keep this to a minimum.  This step is the small bit of applicationContext.xml configuration work we need to do for each controller, we expect it to be less work then reslet given the use of annotations.
  5. Reference Documentation Generation: We are looking into a tool called swagger for documentation generation. Our current reference documentation only lists each end-point (and does not provide information on the request and response expected – leaving users to read the examples or try out the api in an ad-hoc fashion). See screen snap below, our initial experience is positive, but the amount of work required is intimidating.
  6. Updated examples for cURL and Python: We would like to rewrite our examples in a more orderly fashion to make sure both XML and JSON sample requests and responses are provided. Ideally we will inline the “reference files” from the JUnit regression test in step 1 to ensure that the documentation is both accurate and up to date.

You can see a pretty even split in our priorities between performing the migration, and updating the documentation. We believe both of these goals need to be met for success.

Next stop Tuscany

Although this blog post focuses on the sponsorship/planning/logistics side of setting up a code sprint there is one group without whom this event could not happen – our sprint participants and in-kind sponsors (providing a venue & staff).

Thanks to GeoSolutions for hosting us, and to Astun, Boundless, GeoSolutions for the hands-on participation that makes this sprint possible

For more information:

Categories: OSGeo Planet

gvSIG Team: Geoprocesamiento desde Scripting en gvSIG. Vídeo-tutorial disponible.

OSGeo Planet - Thu, 2017-03-09 10:01

Tras la publicación del webinar “Aprende a programar en gvSIG en media hora” os presentamos un complemento perfecto “Geoprocesamiento desde Scripting en gvSIG”.

Por Geoprocesamiento entendemos a las operaciones de tratamiento o manipulación de datos espaciales realizadas en un Sistema de Información Geográfica. gvSIG, con más de 350 geoprocesos, tiene un potencial enorme como software de geoprocesamiento. Potencial que puede ampliarse gracias al scripting.

En este nuevo webinar podréis aprender a acceder desde scripting a los distintos geoprocesos de gvSIG (y de otras librerías) por medio de una librería denominada gvPy, ejecutar geoprocesos con una simple línea de código, convertir modelos de geoprocesos en scripts,…y todo ello lo veréis -tras una breve introducción teórica- mediante ejercicios prácticos.

Si hemos despertado vuestro interés, seguid leyendo…

Filed under: gvSIG Desktop, spanish Tagged: geoprocesamiento, geoprocesos, gvPy, python, scripting, webinar
Categories: OSGeo Planet
Syndicate content