New MapProxy 1.10.0 release
- 2017-05-18 17:00:00+02:00
We are pleased to announce the release of MapProxy 1.10.0. It contains a lot of major and minor improvements.
The latest release is available at: https://pypi.python.org/pypi/MapProxy
To upgrade within your virtualenv:
$ pip install --upgrade --no-deps MapProxy
Updated documentation is available at: https://mapproxy.org/docs/1.10.0/
Support for new cache backends
MapProxy now supports storing tiles in
- Amazon S3 cloud storage
- GeoPackage files
- Redis in-memory databases and
- ArcGIS Compact Cache files.
See: https://mapproxy.org/docs/1.10.0/caches.html#caches
Improved coverages
Areas outside a source coverage can be converted to a transparent background (clipping). It is now also possible to build the union, intersection or difference between multiple coverages.
See https://mapproxy.org/docs/1.10.0/coverages.html#clipping and https://mapproxy.org/docs/1.10.0/coverages.html#union
MapProxy background seeding
The new --duration and --reeseed-interval options of mapproxy-seed allow you to run the seeding for a limited time (e.g. only at night).
See https://mapproxy.org/docs/1.10.0/seed.html#example-background-seeding
Other fixes and changes
There are a lot more changes and improvements including improved ArcGIS REST sources, performance improvements for SQLite and MBTile caches and native GeoJSON support.
For a complete list of see: http://github.com/mapproxy/mapproxy/blob/1.10.0/CHANGES.txt
New MapProxy 1.9.0 release
- 2016-07-22 12:00:00+02:00
We are pleased to announce the release of MapProxy 1.9.0. It contains a lot of major and minor improvements.
The latest release is available at: https://pypi.python.org/pypi/MapProxy
To upgrade within your virtualenv:
$ pip install --upgrade --no-deps MapProxy
Updated documentation is available at: https://mapproxy.org/docs/1.9.0/
Support for ArcGIS REST services
You can now directly integrate ArcGIS services without the need to enable WMS support.
See: https://mapproxy.org/docs/1.9.0/sources.html#arcgis-label
Band merging
The new band merge feature allows you to create false-color or grayscale images on the fly, by selecting different sources for each (color) band.
See https://mapproxy.org/docs/1.9.0/configuration.html#band-merging and https://mapproxy.org/docs/1.9.0/configuration_examples.html#create-grayscale-images
Other fixes and changes
There are many more changes and improvements. For a complete list of see: http://github.com/mapproxy/mapproxy/blob/1.9.0/CHANGES.txt
New MapProxy 1.8.0 release
- 2015-05-18 13:45:00+02:00
We are pleased to announce the release of MapProxy 1.8.0. It contains a lot of major and minor improvements.
The latest release is available at: http://pypi.python.org/pypi/MapProxy
To upgrade within your virtualenv:
$ pip install --upgrade --no-deps MapProxy
Updated documentation is available at: https://mapproxy.org/docs/1.8.0/
Python 3
The most important change is the support for Python 3. MapProxy 1.8 now officially supports Python 2.7, 3.3 and 3.4. Python 2.6 should still work as expected, but is no longer officially supported.
The road to MapProxy 2.0
MapProxy 2 will make a few changes to the configuration. The biggest changes are a few default values that will be changed to more consistent or more modern values. For example, the default tile origin will be compatible with WMTS and EPSG:900913 will no longer be one of the default SRS.
MapProxy 1.8 now prints out warnings if your configuration relies on one of these default values. You will need to address these warnings before upgrading to MapProxy 2.0 in the future.
Other fixes and changes
There are many more changes and improvements. For a complete list of see: http://github.com/mapproxy/mapproxy/blob/1.8.0/CHANGES.txt
Web-based configuration for MapProxy
- 2015-05-06 13:30:00+02:00
We are pleased to announce the development of MapProxy WebConf.
MapProxy WebConf is a small web-based configuration tool for MapProxy. The current implementation of MapProxy WebConf is able to create new configurations and to save them as .yaml files. It already allows easy configuration with drag & drop. Most MapProxy options are available in WebConf and you can edit missing options with the edit manually mode.
MapProxy WebConf is Open Source and released under the Apache Software License 2.0.
The current version shows what is possible with a sophisticated configuration tool. The integration in MapProxy, the configuration and management of seeding-tasks and the loading of existing configurations are a few larger open issues where we are looking for contributions in form of feedback, code or sponsoring.
Feel free to test the tool on our on-line demo: http://webconf-demo.mapproxy.org or check out the Source Code on GitHub
Additional links
New MapProxy 1.7.0 release
- 2014-07-07 16:45:00+02:00
We are pleased to announce the release of MapProxy 1.7.0. It contains a lot of major and minor improvements.
The latest release is available at: http://pypi.python.org/pypi/MapProxy
To upgrade within your virtualenv:
$ pip install --upgrade --no-deps MapProxy
Updated documentation is available at: https://mapproxy.org/docs/1.7.0/
New features
Some noteworthy improvements since 1.6.0 are:
New autoconf tool
There is a new autoconf tool that is able to create MapProxy configurations based on existing WMS services. It reads WMS capabilities documents and creates sources and caches for all layers. It's a great way to get started with a new service.
See: https://mapproxy.org/docs/1.7.0/mapproxy_util_autoconfig.html
Improved error handling in mapproxy-seed
Some services you need to seed are not as robust as you like. We improved mapproxy-seed
to be more resilient with unstable sources. It prints more useful error messages and retries
to request images for longer: seeding will not abort if a source is down for a few hours.
Python 2.5
MapProxy 1.7.0 is the last release with support for Python 2.5.
Other fixes and changes
There are many more changes and improvements. For a complete list of see: http://github.com/mapproxy/mapproxy/blob/1.7.0/CHANGES.txt
New MapProxy 1.6.0 release
- 2013-09-05 11:00:00+02:00
We are pleased to announce the release of MapProxy 1.6.0. It contains a lot of major and minor improvements.
The latest release is available at: http://pypi.python.org/pypi/MapProxy
To upgrade within your virtualenv:
$ pip install --upgrade --no-deps MapProxy
Updated documentation is available at: https://mapproxy.org/docs/1.6.0/
New features
Some noteworthy improvements since 1.5.0 are:
New decorate image API
There is a new API to decorate or manipulate the images returned by MapProxy. It allows you to apply image filters or to add watermarks. The API is similar to the security API.
See: https://mapproxy.org/docs/1.6.0/decorate_img.html
New SQLite and Riak cache backends
There are two new cache backends. Riak is a distributed key-value store that can be used to create high-available caches. The new SQLite cache is similar to MBTiles, but it adds timestamps for proper tile expiration and it uses one database file for each level. This makes it easy to remove complete levels without needing to VACUUM the database.
See: https://mapproxy.org/docs/1.6.0/caches.html
Improved image processing
MapProxy now supports Pillow the successor of PIL. Pillow contains improvements in PNG encoding performance and support for alpha image composition, which improves overlays of multiple transparent layers.
Usability improvements
There are multiple smaller changes that improve the overall usability
of MapProxy. The deployment documentation was updated and
mapproxy-util create -t base-config now creates a minimal MapProxy
configuration and a full configuration with nearly all available options.
Most configuration erros should now give a friendly error message instead
of a stack trace.
Other fixes and changes
There are many more change, improvements and changes. For a complete list of see: http://github.com/mapproxy/mapproxy/blob/1.6.0/CHANGES.txt
3rd anniversary of MapProxy
- 2013-03-22 11:00:00+01:00
Today we are celebrating the third anniversary of MapProxy.
MapProxy grew from a simple tile server with WMS support to a powerful and flexible proxy for maps. Features like the security API, the ability to reproject tiles, support for coverages from Shapefiles or PostGIS and the various tools are just a few things that we couldn't have imagined to have in MapProxy when we were thinking about the future of MapProxy three years ago.
You will get an impression of the development by comparing the configuration options from the first open source release to the latest version.
With open source it is hard to tell how many users MapProxy gained in this time, but we do know that it is now used in countless projects – by federal or state agencies and institutions, by universities, students and hobbyists, by small, national and international companies – all around the world. We also know that there are 200 subscribers on the mailing list and that we get hundreds of downloads each month.
While most of the development, support and release management was made by Omniscale, all this would not have been possible without the support from various companies and institutions who funded large parts of the development of MapProxy.
We would like to thank:
- Blackbridge Geomatics, Canada
- Cadastral Agency of the Hanseatic City of Rostock, Germany
- Camptocamp SA, Switzerland
- Dienstleistungszentren Ländlicher Raum, Rhineland-Palatinate, Germany
- Federal Agency for Cartography and Geodesy, Germany
- GeoInformation Bremen, State Office, Germany
- Norwegian Meteorological Institute
- swisstopo, Federal Office of Topography, Switzerland
- Tike, Information Centre of the Ministry of Agriculture and Forestry, Finland
- ...
And of course, we also like to thank all contributers of MapProxy and all users that gave us valuable feedback on the mailing list, on IRC or during hands-on workshops.
The next release is coming and the list of ideas for 2.0 is growing.
On to the next three years!
New MapProxy 1.5.0 release
- 2012-12-05 11:00:00+01:00
We are pleased to announce the release of MapProxy 1.5.0. It contains a lot of major and minor improvements.
The latest release is available at: http://pypi.python.org/pypi/MapProxy
To upgrade within your virtualenv:
$ pip install --upgrade --no-deps MapProxy
Updated documentation is available at: https://mapproxy.org/docs/1.5.0/
New features
Some noteworthy improvements since 1.4.0 are:
Clipping WMTS/TMS/KML requests to polygons
The security API now also supports limiting requests to polygon geometries for all tile services.
See: https://mapproxy.org/docs/1.5.0/auth.html#limited-to
Mixed image mode
The mixed image mode automatically chooses between PNG and JPEG format depending if the image has transparency or not. This is a great feature for aerial images with transparency at the image boundaries. Thanks to Thomas Bonfort for the idea.
See: https://mapproxy.org/docs/1.5.0/configuration.html#format
Caches as source for other caches
You can now use a cache as a source for another cache. You can use this to directly reproject tiles or to make changes to your grid configuration without losing your existing cache.
See: https://mapproxy.org/docs/1.5.0/configuration_examples.html#using-existing-caches and https://mapproxy.org/docs/1.5.0/configuration_examples.html#reprojecting-tiles
Changed default SRS in WMS
The WMS now supports EPSG:3857, the official EPSG code of the Webmercator projection (EPSG:900913). UTM 32/33N and Gauß-Krüger zones 3-5 were removed from the default list, since they are only usable in parts of Europe. Make sure you've configured all SRS you are using in the service configuration.
New tools
There are three new tools:
mapproxy-util grids: debug and analyze grid configurationmapproxy-util wms-capabilities: get bbox and min/max_scales of existing WMSmapproxy-util export: export (parts) of your cache to MBTiles or a TMS directory
See: https://mapproxy.org/docs/1.5.0/mapproxy_util.html
Other features
- use_grid_names option to access Tiles/TMS/KML layers by grid name instead of EPSGXXXX
- origin option for TMS to change default origin of the /tiles service
- continue stopped/interrupted seed processes
- support min_res/max_res for tile sources
Improvements
- do not show layers with incompatible grids in WMTS/TMS demo
- make 0/0/0.kml optional for the initial KML file
- use BBOX of coverage for capabilities in seed_only layers
- ignore debug layer when loading tile layers
- simplified coverage configuration
- add reloader option to make_wsgi_app()
- add MetadataURL to WMS 1.1.1 capabilities
- improved WMTS services with custom grids (origin)
- use in_image exceptions in WMS demo client
- larger map in demo client
- always request with transparent=true in WMS demo client
- use in_image exceptions in WMS demo client
Other fixes and changes
For a complete list of changes see: https://github.com/mapproxy/mapproxy/blob/1.5.0/CHANGES.txt
New MapProxy 1.4.0 release
- 2012-05-15 15:00:00+02:00
We are pleased to announce the release of MapProxy 1.4.0. It contains major and minor improvements.
The latest release is available at: http://pypi.python.org/pypi/MapProxy
To upgrade within your virtualenv:
$ pip install --upgrade --no-deps MapProxy
Updated documentation is available at: https://mapproxy.org/docs/1.4.0/
New features
Some noteworthy improvements since 1.3.0 are:
Clipping requests to polygons
The security API now supports limiting requests to polygon geometries with the limited_to feature.
See auth API.
Demo service
The demo service now allows you to query WMS layers in all supported SRS. It now also supports WMTS if you have enabled this service.
Other improvements
- WMS 1.3.0 now supports extended layer capabilities for metadata etc.
- You can configure fallbacks for errors in tile sources. MapProxy can create an empty tile if the tile source returns a 404 not found for a tile, for example.
Note: There was a bug in the configuration loader that permitted to have an error in the layers configuration. Please make sure your configuration is valid after upgrading.
Changelog
For a complete list of changes see: http://bitbucket.org/olt/mapproxy/src/1.4.0/CHANGES.txt
New MapProxy 1.3.0 release
- 2012-01-13 14:00:00+01:00
We are ready to announce the release of MapProxy 1.3.0. It contains major and minor improvements.
The latest release is available at: http://pypi.python.org/pypi/MapProxy
To upgrade within your virtualenv:
$ pip install --upgrade --no-deps MapProxy
Updated documentation is available at: https://mapproxy.org/docs/1.3.0/
New features
Some noteworthy improvements since 1.2.0 are:
RESTful WMTS
The MapProxy WMTS now also supports the RESTful API. This service also supports custom URL templates for your service.
See WMTS configuration.
CouchDB cache backend
You can now use a CouchDB as a backend for tile caches. Each cache gets stored into a separate database and you can configure the layout of the URLs of each tile. You can also add additional metadata for each tile.
mapproxy-util serve-multiapp-develop tool
The new command is similar to serve-develop but it starts a
MultiMapProxy instance.
See mapproxy-util documentation.
Other features
-
You can limit the maximum WMS response size in pixels with max_output_pixels.
-
You can configure additional SRS for which MapProxy should add bounding boxes to the WMS capabilities with the new wms.bbox_srs option.
Changelog
For a complete list of changes see: http://bitbucket.org/olt/mapproxy/src/1.3.0/CHANGES.txt
New MapProxy 1.2.0 release
- 2011-08-31 15:00:00+02:00
The 1.2.0 release contained a bug with the watermark configuration. There is already a 1.2.1 release available.
We are finally ready to announce the release of MapProxy 1.2.0. It contains major and minor improvements.
The latest release is available at: http://pypi.python.org/pypi/MapProxy
To upgrade within your virtualenv:
$ pip install --upgrade --no-deps MapProxy
Updated documentation is available at: https://mapproxy.org/docs/1.2.0/
New features
Some noteworthy improvements since 1.1.0 are:
Cache
You can use the MBTiles format for new caches and you can load existing MBTiles cache files.
The directory layout for file based caches is now configurable and you can choose a TMS compatible layout besides the default TileCache compatible one.
See cache configuration.
mapproxy-util scales tool
The new mapproxy-util scales tool helps to convert between scales
and resolutions.
See mapproxy-util documentation.
Other features
-
Mapnik source now supports the selection of single layers.
Changelog
For a more complete list see: http://bitbucket.org/olt/mapproxy/src/1.2.0/CHANGES.txt
New MapProxy 1.1.1 release
- 2011-06-26 10:57:00+02:00
There is a new bug fix release of MapProxy.
The 1.1.1 release fixes:
- add back transparent option for mapnik/tile sources (in addition to image.transparent)
- keep alpha channel when handling image.transparent_color
- fixed combining of multiple WMS layers with transparent_color
- fixed header parsing for MapServer CGI source
The latest release is available at: http://pypi.python.org/pypi/MapProxy
To upgrade within your virtualenv:
$ pip install --upgrade --no-deps MapProxy
New MapProxy 1.1.0 release
- 2011-05-31 10:40:00+02:00
We are finally ready to announce the release of MapProxy 1.1.0. It contains lots of major and minor improvements.
The latest release is available at: http://pypi.python.org/pypi/MapProxy
To upgrade within your virtualenv:
$ pip install --upgrade --no-deps MapProxy
Updated documentation is available at: https://mapproxy.org/docs/1.1.0/
MapProxy 1.1.0 is the first version that is released under the Apache Software License 2.0. This replaces the old GNU AGPL License terms with a more open and liberal license.
New features
Some noteworthy improvements since 1.0.0 are:
New sources
There are two new sources that directly integrate Mapserver and Mapnik into MapProxy. There is no need to setup an extra WMS server for these map services.
See MapServer source and Mapnik source documentation.
The tile source was extended by the arcgiscache_path and bbox
parameters. arcgiscache_path allows the integration of existing ArcGIS
tile caches (L09/R05397fb1/C0012d687) and the bbox parameter is useful
for WMS-C services that expect a fixed parameter order.
See tile source documentation.
Tagged WMS Sources
You can tag WMS source names with layer names. It is no longer required to specify a WMS source multiple times, for each layer combination you use. You can now define a WMS once without any layers and then tag the source name with the layers you need.
sources:
wms1:
type: wms
req:
url: http://example.org/service?
caches:
mycache1:
sources: ['wms1:lyr1,lyr2']
grids: [mygrid]
mycache2:
sources: ['wms1:lyr3,lyr4']
grids: [mygrid]
See WMS documentation.
Configurable Image Formats
Image formats are now much more flexible. You can now create custom image formats and modify them for each source or cache individually. The configuration of paletted PNGs or the JPEG quality is no longer a global option.
See image options.
There is now also support for integer images, e.g. for DEMs.
WMTS
There is first support for the OGC WMTS standard. It implements KVP requests and can be used with existing caches.
See [WMTS service documentation] (https://mapproxy.org/docs/1.1.0/services.html#web-map-tile-services).
Improved logging
The logging was improved and unified. System information is now logged at
mapproxy.system, configuration errors at mapproxy.config and all
source errors under mapproxy.source.wms, etc.
All requests that MapProxy makes are now logged to
mapproxy.source.request. The request log format changed and it now
contains the complete URL (easy to copy&paste), the size and the duration
of the request. The log contains also requests to non-HTTP sources like
Mapnik or Mapserver.
Improved configuration loading
The configuration loading is now more robust. You shouldn't see any stack traces when your configuration contains any syntax errors. The MapProxy and the seed configurations are now validated and you should see detailed information about missing or unknown options.
Cache location
The default location of the cache will change with the next release (1.2). The
current default is ../var/cache_dir which is appropriate for the
configuration from paster create, but not for the configuration from
mapproxy-util create (see below). You should configure
globals.cache.base_dir with your cache directory. Until then you will get
a FutureWarning during startup.
New mapproxy-util command line tool
There is a new mapproxy-util command that replaces the paster command for running the development server. You can now start a test server with:
mapproxy-util serve-develop mapproxy.yaml
Paster (PasteDeploy/PasteScript) is no longer a requirement, but it is still supported.
Changelog
For a more complete list see: http://bitbucket.org/olt/mapproxy/src/1.1.0/CHANGES.txt
New MapProxy 1.0.0 release
- 2011-03-03 11:11:11+01:00
We are finally ready to announce the release of MapProxy 1.0.0. It contains lots of major and minor improvements.
The latest release is available at: http://pypi.python.org/pypi/MapProxy
To upgrade within your virtualenv:
$ pip install -U --no-deps MapProxy
Updated documentation is available at: https://mapproxy.org/docs/1.0.0/
Upgrade notes
A bug, introduced in a previous release, prevented that the tile locks get removed. This issue is solved since 1.0.0b2 and MapProxy will remove existing lock files on the first access to the cache. This might take a while if the directory already contains a lot of lock files (probably thousands and more).
We suggest that you remove the complete lock directories by hand before upgrading:
rm -r cache_dir/*/tile_locks
New features
Some noteworthy improvements are:
mapproxy-seed
mapproxy-seed got a new, flexible configuration format (the old format is
still supported). You can now define multiple seed and cleanup tasks and call
each separately from the command line interface (--seed and --cleanup
options).
It is now possible to reseed specific areas without triggering any cleanup.
There are also two new --summary and --interactive options that help
to understand what the seed tool will do.
See: https://mapproxy.org/docs/1.0.0/seed.html
FeatureInfo XSL transformations
MapProxy now supports content aware merging of multiple HTML/XML feature responses and it supports XSL transformations. See: https://mapproxy.org/docs/1.0.0/configuration_examples.html#featureinformation
Image manipulation
You can now make non-transparent layers transparent. Either with an opacity value, with blends the layer over the others, or with a color value that should be converted to transparent. See: https://mapproxy.org/docs/1.0.0/sources.html#image-transparent-color
Authentication
There is a new powerful authentication interface. See: https://mapproxy.org/docs/1.0.0/auth.html
Changelog
For a more complete list see: http://bitbucket.org/olt/mapproxy/src/1.0.0/CHANGES.txt
Final note
We also like to use this announcement to say thank you to the ever growing community. To everyone that helped funding the many new features and to everyone that offered help, gave feedback or talked/blogged/tweeted about MapProxy.
Thank you!
New MapProxy 0.9.1 release
- 2011-01-10 17:00:00+01:00
We are finally ready to announce the release of MapProxy 0.9.1. It contains lots of major and minor improvements.
The latest release is available at: http://pypi.python.org/pypi/MapProxy
To upgrade within your virtualenv:
$ pip install -U MapProxy
Updated documentation is available at: https://mapproxy.org/docs/0.9.1/
Some noteworthy improvements are:
Coverage support
You can limit any tile/WMS source to the actual coverage of the data. The configuration is similar to the seeding tool. You can use a simple BBOX, WKT files or OGR datasources like Shapefiles to load the boundaries of your data. Requests for data outside of the coverage will result in empty images and the source will not be requested.
See: https://mapproxy.org/docs/0.9.1/coverages.html
WMS improvements
We added lots of new features to the WMS implementation of MapProxy. Some of them:
- Nested layer groups
- WMS 1.1.0 support
- GetLegendGraphic support
- ScaleHint support
- SLD support for WMS sources
MultiMapProxy
Use multiple MapProxy configurations in a single MapProxy instance: https://mapproxy.org/docs/0.9.1/deployment.html#multimapproxy
Changelog
For a more complete list see: http://bitbucket.org/olt/mapproxy/src/0.9.1/CHANGES.txt
Nightly documentation
- 2011-01-07 14:15:00+01:00
We integrated the documentation building into our testing system and as a result we are now able to provide up-to-date documentation of MapProxy.
You now find the latest documentation of the development trunk at: https://mapproxy.org/docs/nightly/
The official documentation of the latest release is still available at the old location.
New MapProxy 0.9.0 release
- 2010-10-18 00:00:00+02:00
We are finally ready to announce the release of MapProxy 0.9.0. It contains lots of major and minor improvements.
The latest release is available at: http://pypi.python.org/pypi/MapProxy
To upgrade within your virtualenv:
$ pip install MapProxy
Updated documentation is available at: https://mapproxy.org/docs/0.9.0/
Some noteworthy improvements are:
Demo Service
MapProxy now comes with a demo service that lists all configured WMS and TMS layers. You can test each layer with a simple OpenLayers client at /demo.
https://mapproxy.org/docs/0.9.0/services.html#mapproxy-demo-service
Configuration
A biggest and most apparent change is the configuration format. MapProxy still uses YAML as a configuration format, but the structure changed a bit.
Some benefits of the new configuration: Most global options, like image resampling and meta_size, are now configurable for each cache; you can now reuse parts of the configuration, like grid or source definitions; and there is now a single configuration file instead of two (service.yaml and proxy.yaml).
There is a small migration guide that should help to update your configuration. https://mapproxy.org/docs/0.9.0/migrate.html
Oh, and in case you didn't know, YAML is compatible with JSON. So you can use JSON for the configuration if you like.
Less dependencies
We removed two dependencies that required a C compiler. The only dependency that still requires compilation is the Python Image Library (PIL). If you use PIL from your system distribution (python-imaging on Debian/Ubuntu) you will no longer need a compiler during installation.
We switched the templates away from Jinja2 to Tempita, a minimal Python-only template engine. It is a bit slower, but it is only used for capability documents, so the WMS and TMS performance is not affected.
The other dependency which required compilation was pyproj. We we added a wrapper to MapProxy that directly uses libproj, the Proj4 C-library. libproj is available on most systems and should already be installed when you use software like MapServer or PostGIS. There is a fallback to pyproj, i.e. you can still use the binary distribution of pyproj on Windows.
New MapProxy 0.8.5 release
- 2010-09-02 00:00:00+02:00
We are proud to announce the release of MapProxy 0.8.5. Read on for more information.
The latest release is available at: http://pypi.python.org/pypi/MapProxy
To upgrade within your virtualenv:
$ pip install --no-deps --upgrade 'MapProxy<=0.8.99'
Note: The '<=0.8.99' prevents from installing 0.9 when it is released.
Updated documentation is available at: https://mapproxy.org/docs/0.8.5/
Release information
The 0.8.5 release contains minor bug fixes and improvements. Some of the features of the new release are:
- mapproxy-seed:
- fixed libgdal loading on some Linux systems
- check for intersections with seed shapes on all levels
- add origin options to /tiles service to support Google Maps clients
- now Python 2.7 compatible (some tests failed before)
Improving the performance for PNG requests
- 2010-08-02 00:00:00+02:00
MapProxy uses the Python Image Library (PIL) for everything image related: decoding, resizing, cropping, merging, transforming and encoding. Core parts of PIL are written in C and so is MapProxy able to keep up with pure C servers, and even trump them. Only the PNG encoding is significantly slower than other implementations.
There are two major variations of PNG files, often called PNG-24 and PNG-8. PNG-24 stores full color information for each pixel, while PNG-8 only supports a total of 256 different colors per image. The benefit of PNG-8 is that the files size reduces to 1/3. This process of reducing the possible 16.7 million colors of PNG-24 to a palette of only 256 colors is called color quantization.
We profiled the PNG encoder and found the bottleneck of the encoding process was in that color quantization. We've implemented a new algorithm, based on the octree algorithm, that offers better performance. Much better performance. The whole encoding process itself got about 10 times faster, increasing the MapProxy performance for PNG requests about 2-3 times, wich is now on a par with the JPEG format.
It even gets better: The new quantizer has also full transparency support. Prior to this, MapProxy could return images with transparency only in PNG-24 mode, but now transparency is also supported for PNG-8. For these images, the file size reduces to 1/4 of the PNG-24 size, while having the same performance boost.
How to use the new quantizer
The development of our modification is available at http://bitbucket.org/olt/pil-117-fastpng The modification is not yet included in the official PIL version, but the maintainer told me that it should be merged later this summer and that it should be available in the next PIL release.
But you can already use it anyway. There is a source package available and you can install and compile it with:
$ pip install http://bitbucket.org/olt/pil-117-fastpng/downloads/PIL-1.1.7-fastpng-a6.tar.gz
You need the latest MapProxy 0.8.4 release or the upcoming 0.9.0 to get support for the new quantizer.
By default, MapProxy will still generate PNG-24 for transparent images. To enable PNG-8 for these images, you have to add the following to your proxy.yaml:
image:
paletted: true
Thats it. You should now have a MapProxy that is 2-3 times faster for PNG files, and with the reduced file size even faster for you users.
Have fun.
New MapProxy 0.8.4 release
- 2010-08-02 00:00:00+02:00
We are proud to announce the release of MapProxy 0.8.4. Read on for more information.
The latest release is available at: http://pypi.python.org/pypi/MapProxy
To upgrade within your virtualenv:
$ pip install --no-deps --upgrade 'MapProxy<=0.8.99'
Note: The '<=0.8.99' prevents from installing 0.9 when it is released.
Updated documentation is available at: https://mapproxy.org/docs/0.8.4/
Release information
The 0.8.4 release contains minor bug fixes and improvements.
Some of the features of the new release are:
- Improved PNG performance with PIL fastpng branch.
- New concurrent_requests option to limit requests for each source WMS server.
- Extra newline at the end of all templates. Some deployment setups removed the last characters.
MapProxy Raster Image Benchmark
- 2010-06-24 00:00:00+02:00
We did some benchmarks of MapProxy to show you how it compares to other WMS servers when it comes to serving raster images. This is a follow up to the WMS benchmark presented at the FOSS4G in Sydney and the additional tests done by Chris Tweedie.
We did not re-run all tests, but we have included MapServer as a comparison. You can use the results as a baseline to compare the MapProxy results with the numbers from the other benchmarks.
If you are not familiar with MapProxy, you should note that it is not a WMS server like MapServer or GeoServer, but a caching proxy for existing WMS servers. However, MapProxy does not cache every single response like an HTTP proxy such as Squid does. It builds up an internal tile cache and uses this cache to create new WMS responses. For raster images it basically does the same as a regular WMS server does: read raster data, resample the data to the right output resolution and deliver the result.
In this test we used the MapServer installation as the data source of the MapProxy.
Configuration
The test server is a virtualized XEN machine with 4 CPU cores and 7GB RAM running Ubuntu 9.10. The host machine has an Intel Xeon X3360 CPU with 4 cores at 2.83GHz. The test server was the only VM that was running during the benchmarks.
MapProxy
We used MapProxy 0.8.3 running with Python 2.6.4. It was deployed with gunicorn behind nginx as a reverse-proxy (no caching enabled). Gunicorn was configured to run 6 worker processes of MapProxy.
MapServer
We used MapServer 5.6.3. It was deployed as a FastCGI server behind Apache 2.2.12. We used 12 worker processes, which offered the best performance on this machine.
Test data
We used a Blue Marble Next Generation image as a data source for our tests. The image covers the whole World and is 21600 x 10800px. We used an uncompressed tiled TIFF with overviews (10800x5400, 5400x2700, 2700x1350, 1350x675). This should compare to the BigTiff dataset in the FOSS4G benchmark and to the Tiled (internal) from Chris Tweedies tests.
For MapProxy we used JPEG as the internal caching format and a tile size of 256x256 pixels.
Tests
We used the same BBOX extends and output dimensions that were used in the FOSS4G benchmarks. We took the 2000 random extends from bluemarble.csv and build a list of URLs for each tested server.
The image sizes vary between 256x256 and 1024x768 pixel. We tested JPEG as the output format with nearest neighbor resampling.
We tested different concurrency levels to see how the servers react under different loads. Each URL was requested once per test run and each test run was repeated three times. We used the best result of each test run.
Results
Ok, here are the results of our benchmark. At first the results for requests in EPSG:4326, the native reference system of the input data/cache:
MapServer can deliver up to 122 requests per second at 20 and 40 concurrent clients, MapProxy peaks at 40 concurrent clients with 157 requests per second, an increase of ~28%. Both servers can hold the performance with increasing concurrency.
And here are the results for requests in EPSG:900913, also known as the web mercator projection EPSG:3785. We used the same EPSG:4326 cache for MapProxy, so both servers need to reproject from EPSG:4326 to EPSG:900913.
MapServer can deliver up to 74 requests per second from 20 to 150 concurrent clients, MapProxy delivers 113 requests per second from 20 to 80 clients, an increase of ~52%.
Both servers showed very good results, and they exceeded our expectations.
Comparing with FOSS4G benchmarks
The MapServer results are much better than the results from the last FOSS4G benchmarks, 122 req/s to 28 req/s peak. There are two explanation for the difference. First, our test data set is only 900MB compared to the 16GB of images they used. We guess that the FOSS4G servers had some disk I/O during the tests that slowed the results down. Second, our test server is more recent and has more power.
Comparing with Chris' benchmarks
This one is more interesting. The CPUs are not that different, but his machine has two CPUs -- 8 cores. Still we got nearly three times as may requests per second. So either something was misconfigured on Chris's server or MapServer performs way better on Linux than on Windows.
Conclusion
You have to consider that both servers do the same job. As we already stated at the beginning, they just read raster data, resample the data to the right output resolution and deliver the image to the client. Therefore it is great to see that MapProxy can beat the oldtimer MapServer.
But is it worth to use MapProxy for a ~25-50% performance boost? Maybe not, but this was a simple test case. Serving raster images is the easiest job for MapServer, compared to rendering vector data from a shapefile or database. Rendering small scale maps with lots of data takes significant longer, render times of few seconds are not uncommon. The performance of MapProxy, on the other hand, is independent of the data. So, while the benefits of MapProxy seam small in this test, they can become huge when you start to serve larger raster or vector datasets.
Come join our mailing list if you want to discuss the results, or leave a comment below.
