June 30, 2012
Maperitive: Alpenglow Effect Using A Custom Shader

The latest release comes with a new Python script called Alpenglow.py which renders a hillshading relief adding some yellowish tint to illuminated high slopes to enhance the 3D effect. It was inspired by the “See the light: How to make illuminated shaded relief in Photoshop 6.0" article on the excellent shadedrelief.com site.

You can run the script by executing

run-python Samples/Python/Alpenglow.py

The script generates the hillshading for a fixed map area in Switzerland, but you can change this quite easily in the script. You can also change other parameters, like shading (and alpenglow) colors, elevation ranges etc.

The script also demonstrates the power of writing a custom shader. A shader transforms a small part of the digital elevation model (DEM) into a pixel color, based on the elevation, aspect and the slope of that part of the DEM. This way you can write your own hillshading effects function and then supply it to the generator. Here’s how it’s done for Alpenglow.py:

# Execute the hillshading
bitmap = ReliefUtils.generate_relief_bitmap(dem, relief_bbox, Srid.WebMercator, 1, alpenglow)

The last parameter of the generate_relief_bitmap function is the shader function, in our case called alpenglow:

# The actual shader function which is called for each pixel of the hillshading bitmap.
# It receives the elevation, the aspect (orientation) of the slope and the steepnes (slope).
def alpenglow(elevation, aspect, slope):
    sin_slope = math.sin (slope)

    nx = math.cos(aspect) * sin_slope
    ny = math.sin(aspect) * sin_slope
    nz = math.cos(slope)

    direction = dotproduct(nx, ny, nz, lx, ly, lz)

    direction = (direction - minDirection) / (1 - minDirection);
    if direction < 0:
        direction = 0

    # This is how we can mix two colors together.
    shade_color = shade_base_color.mix (light_color, direction)

    if direction > 0.0 and elevation > min_alpenglow_elevation:
        elevation_factor = min((elevation - min_alpenglow_elevation) / delta_alpenglow_elevation, 1);

        alpenglow_factor = math.pow(direction, 4)
        return shade_color.mix (alpenglow_color, elevation_factor * alpenglow_factor).argb;

    return shade_color.argb;

May 24, 2012
Mercator’s Egg

UPDATE: my math was buggy, the eggs look a bit different after the correction. Note that the formula fails on very large distances (> 2000 km).

A little Maperitive Python script I was playing with today, showing the line of distance of 3000 km (larger egg) and 2000 km (smaller egg) from my home town of Maribor, drawn on the standard Web Mercator projection map:

Here’s the script:

from maperipy import *
import math

location = Point(15.6530555556, 46.5575)
distance = 3000000

layer = Map.add_custom_layer()

symbol = PolygonSymbol("circle", Srid.Wgs84LonLat)
symbol.style.pen_width = 8
symbol.style.pen_color = Color("blue")
symbol.style.pen_opacity = 0.5
symbol.style.fill_opacity = 0

earth_circumference = 6371000 * math.pi * 2;
points = []
arc = distance / earth_circumference * 360;

for angle in range(0, 360):
    radians = math.radians(angle)
    y = location.y + math.sin(radians) * arc
    lat_factor = math.cos(math.radians(y))
    x = location.x + math.cos(radians) * arc / lat_factor
    points.append(Point(x,y))

circle = Polygon(LinearRing(points))
symbol.add(circle)

layer.add_symbol(symbol)

May 4, 2012
Oxford Bus & Cycle Map in print

I’ve just received the “Oxford Bus & Cycle Map” from Richard Mann (thanks, Richard!). The interesting thing is that it was made from OpenStreetMap data using Maperitive. It’s a stunning beauty, Richard has put a lot of effort into it.

Online versions of the maps are here: http://www.transportparadise.co.uk/maps/

May 4, 2012
Results Of The Maperitive Survey

Last week I posted a quick survey for Maperitive users. The basic motivation was to see how many people are still using Windows XP SP2, so I can decide whether I can migrate Maperitive to .NET 4.0 (which is not available on SP2). But I added a few more questions in the mix, just for the fun of it.

I got 138 reponses so far, which is much more than I expected, so thanks for everyone who participated.

The results (I just copy&pasted charts from Google Docs, sorry for the labels cut-off):

What operating system(s) are you using to run Maperitive?

Participants could choose multiple answers - in fact about 1/4 of participants were using Maperitive on multiple operating systems. About 4% of people marked they are using Windows XP SP2 - I am wondering what is the reason they haven’t upgraded to SP3 (my guess is they are using Maperitive in corporate environments which tend to be slow on upgrades).

Two things that I think are important from these results:

  1. I can start work on migrating to .NET 4.0. For those unfortunate few who will be left out by this I can only say: try to upgrade to Service Pack 3. The official .NET 4.0 has been released two years ago (and first beta was released back in 2009) and it’s increasingly hard to support development on older versions of .NET framework. 3rd party libraries Maperitive depends on have mostly migrated to .NET 4.0 and this means missing out on new features and bugfixes.
  2. A lot (50%) of people are using Maperitive on Linux and Mac. This is much more than I expected and it means I will still have to keep Maperitive running on Mono. This is sometimes hard, since not everything is supported by Mono correctly. It also means a lot of people are willing to put up with an inferior user interface (due to Mono porting issues) for the sake of the functionality Maperitive offers. This is complimentary.

What it is the main reason you’re using Maperitive?

No surprises here, other than that at least 1/3 of users find Maperitive simple to use. This is a high number, given the fact that the documentation is not that good.

There’s one interesting (and a little bit surprising) answer:

We can use OSM offline, without prerendering thousands of tiles. It is very helpful for emergency services, because you cannot rely on a mobile internet connection everywhere or in the case of a disaster.

What kind of a user do you consider yourself to be?

Looks like the main audience of Maperitive are users who like to hack. Only 18% of participants in the survey consider themselves to be non-technical. I guess that explains the large “simple to use” percentage in the previous question :)

I use Maperitive mainly to…

Another question with multiple possible answers. I’m pleased that a lot of you are using Maperitive’s “power tools” like tile generation and exporting. I’m also pleased that at least 37% of participants are using the SVG export functionality, which I worked hard to implement.

I use Maperitive mainly for…

I guess I shouldn’t be surprised that most of you are doing hobby mapping with Maperitive. As OpenStreetMap reaches more people (and companies), I think the percentage of non-profit and commercial usages will increase.

Conclusion

The survey is still open (I think I will keep it open for some time), so you’re welcome to have your say. There will be more surveys in the future - this one was done as an experiment, in just 5 minutes. Again, thanks for participating!

10:41am  |   URL: http://tmblr.co/Zm5DjxKre3OP
Filed under: Maperitive survey 
November 28, 2011
Maperitive Beta: Subpixel Accuracy

Even though most of Maperitive work I’m doing lately is done in a separate branch that will come out as a brand new Maperitive 2.0 release some time in the beginning of 2012, I do occasionally implement some quick enhancements to the currently “unofficially” released Maperitive beta. I’ve just uploaded a new beta version with a couple of neat features, one of which I will describe in this post (the other will have to wait for the next post).

Subpixel Accuracy

This is an important, although probably not very well known feature Mapnik provides. What is subpixel accuracy? Simply put, it is the ability to set the thickness of the line in fractions of a pixel (image stolen from AGG’s site):

Up until now this was not really possible in Maperitive. Well, you could specify the thickness as, say, 1.3 pixels, but it would be rendered as a line with thickness of one pixel, ignoring the fraction. The reason is simple: Maperitive uses GDI+ library for rendering, which doesn’t do subpixel accuracy (it does support antialiasing, but that’s not enough). Mapnik, on the other hand, uses superior Anti-Grain Geometry (AGG) library (you can read more about it in AGG’s Introduction page, which is very interesting).

Why is subpixel accuracy important? Certain map features require delicacy. In his Cartographic Relief Presentation book Eduard Imhof gives very precise recommendations for elevation contour lines thicknesses. 0.05 to 0.08 mm for black normal contours for map scale 1 : 50,000, as an example. Without subpixel accuracy it is very difficult to come even close to such measures and then you end up with both normal and index contours (the thicker ones) having the same thickness in the rendering:

So given that I cannot directly render lines with subpixel accuracy and using AGG in Maperitive is not a practical solution because of various technical reasons I won’t go into, I’ve more or less given up on this.

The Solution

… Until I came up with a simple but effective trick to achieve subpixel accuracy: render the whole map scaled to (say) three times and then downsample the whole image back to the “normal” scale. Before I delve into details, here’s how the same map looks like using this technique:

I think the difference is obvious: you can now see two distinctive contour line types. Index lines are 1.25 wide, while normal lines have a width of 0.65 pixels.Don’t worry about these numbers, subpixel accuracy is not that precise, but it is precise enough.

The Details

Subpixel accuracy can be tuned for three Maperitive commands:

  • export-bitmap
  • export-3d
  • generate-tiles

All of them have a new subpixel parameter which accepts a positive integer value. This value is used as a scaling factor when rendering. Once the rendering is done, Maperitive uses Supersampling algorithm to resample (downsample actually) the rendering to the scale factor of 1. So for example if you want to generate tiles using subpixel accuracy, you type in something like

generate-tiles minzoom=10 maxzoom=12 subpixel=3

Caveats

What is the best value for subpixel parameter, you may ask? I’ve been experimenting with the value of 3. You could try larger values, although I doubt there will be a noticeable increase in the quality of rendering. The downside of larger values is that Maperitive takes longer to render. This could considerably slow down your tiles generation, so there is a trade-off between the quality and speed (isn’t there always?). Measurements of tile generation speed show that by increasing subpixel value by 1 halves the speed of tile generation. In future this should be mitigated by allowing Maperitive to render tiles using multiple processors.

You should also be aware that if you intend to use subpixel accuracy for exporting large bitmaps, Maperitive has to actually allocate a much bigger bitmap in the process of rendering. So for example if you want to create a 4000x4000 bitmap using subpixel=3, Maperitive will want to allocate a 12000x12000 bitmap in the process and will probably fail due to limitations of GDI+ bitmap size. So there is no free lunch (at least until I figure a way around this problem ;) ).

One final note: interactive map does not use subpixel accuracy since this would make the map rendering pretty slow. In future versions Maperitive will provide some kind of “map quality” / “level of detail” setting, so you’ll be able to control this.

Enjoy!

October 19, 2011
Maperitive: 3D Export

The latest Maperitive beta (download it from here) now contains a new command called export-3d, which generates a 3D mesh using the digital elevation model and places the map texture on top of it. The 3D model is exported into a COLLADA format, which can be imported into various 3D programs (Google SketchUp is the one I mostly used, it’s free).

Quick How-To

The quickest/easiest way to generate a 3D export is to move the map to the area you’re interested in and simply select the Tools | Export To 3D menu command. Maperitive will then generate files in the output/Maperitive3D directory of your Maperitive installation. If you do not load your own OSM vector layer, Maperitive will use OSM tiles, the same ones used to show tne OSM web map.

NOTE: be careful to keep the map area fairly small, otherwise you’ll end up with a 3D model that you won’t be able to import into other software, especially if you don’t have a top-of-the-line computer at your disposal.

Importing Into SketchUp

After opening SketchUp, choose **File | Import…” menu command. NOTE: Before browsing for the file, click on the Options… button and make sure Validate COLLADA file is turned off:

Validation should be disabled because it makes importing unbearably slow, especially for larger models. After you’ve done this, browse to your generated Maperitive3D.dae file and open it.

Depending on the size of your model, you might have to wait for some time for the importing to finish. Sometimes I even have to resort to killing the SketchUp process and regenerating a simpler model.

Configuring SketchUp

After the model has been successfully imported, you may want to tweak a few settings in SketchUp to make the rendering look better:

  • View | Edge Style | Edge - uncheck this so the edges of surfaces are not shown.
  • View | Face Style - choose Shaded With Textures to get the best effect.
  • View | Shadows - check this to turn on the sun shadows.

Tweaking The 3D Model

export-3d command provides additional parameters we can tweak. Type in help-command export-3d into the command prompt and you’ll get the list of those parameters:

COMMAND NAME: export-3d 
DESCRIPTION: generates a 3D map from the current map view and saves it to the disk 
PARAMETERS: 
  output-dir=<the directory where output files will be saved> (text, optional)
  mesh-points=<the maximum number of points the terrain TIM mesh should have> (integer, optional)
  tin-error=<the maximum allowed elevation difference (in meters) when simplifying the terrain TIN mesh (default is 1 meter)> (real number, optional)
  width=<bitmap width> (value, optional)
  height=<bitmap height> (value, optional)
  map-scale=<map scale to use when exporting> (value, optional)
  scale=<graphics scale to use when exporting> (value, optional)
  dpi=<DPI to use when exporting> (value, optional)
  zoom=<zoom level to use when exporting> (value, optional)

Better Bitmap Texture

By setting various bitmap parameters you can increase the resolution of the bitmap and/or the zoom level of the underlying OSM bitmap.

TIN Tweaks

To increase performance, Maperitive uses Garland-Heckbert’s TIN simplification algorithm to simplify the terrain triangulated irregular network (TIN). There are two command parameters that affect the simplification process:

  • tin-error: this is the maximum error (tolerance) allowed between the actual DEM elevation and the generated model, in meters. The default value is 1 meter and if your model becomes too big to handle, I suggest increasing the tolerance value.

  • mesh-points: this is the maximum number of points (vertices) the TIN should have. The simplification algorithm starts from a very simple model (just two triangles) and incrementally adds new points to it. Upon reaching the maximum number of points, the algorithm stops (regardless of the tin-error setting). This is a safeguard against the model going wild with too many points. So I suggest experimenting with those two values to get something workable for your case.

Setting The Map Area More Precisely

Instead of relying on the map window to act as your area of interest, you can specify printing bounds (right click on the map) and move them to an exact position of your choice.

Final Notes

This 3D export function is by no means of the same quality as some other OSM 3D projects - you don’t get any 3D objects apart from the terrain model, so don’t expect to see any 3D buildings, trees etc. It is, however, a very simple tool to use (I hope) and it produces an output in a fairly open standard (COLLADA) which can then be tweaked with other software.

The next logical step would be to include the things like roads and buildings as actual 3D objects, but I will leave that for the future since my task list already very full with other features.

Please send me feedback if you find this feature useful (or indeed if you find it crappy). Since all this is beta, expect to find bugs. Also, if you’re more of an expert in 3D field than me, a question for you: can you recommend any (preferably free) tool for raytracing which can consume COLLADA files generated by Maperitive?

October 11, 2011
Maperitive: maperipy Progress

WARNING: this is one of those teaser-posts - you get to hear about the cool new features, but you won’t be able to try them out, at least not very soon.

Introduction

In the last few weeks since the beta release I’ve been working on a custom job involving Maperitive. I won’t go into details about the job itself, but the important thing is that this job was an opportunity for maperipy Python API to be extended with some new features. I’ll go through some of them, together with sample Python code.

Geometries

maperipy provides classes for basic geometries (compatible with OGC’s Simple Features Specification) like Point, LineString, LinearRing, Polygon, MultiPolygon etc. I’ve tried to stick to the syntax and semantics used by shapely library (unfortunately shapely itself cannot be used directly because it won’t run in IronPython - it depends on some C code in the backstage, but more about this some other time).

Custom Layers

maperipy now allows users to create their own custom map layers which can be programmatically filled with map symbols using geometries described previously:

# we first create a custom layer on a map
layer = map.add_custom_layer()

# create a simple geometry
line = LineString([(0, 0), (0, 10), (10, 10), (10, 0)])

# add a symbol for it...
symbol = LineSymbol("my_line", (line, ))
# ... and define the visual style
symbol.pen_width = 5
symbol.pen_color = Color("red")

# now we add the symbol to the layer
layer.add_symbol(layer)

Shapefiles

The custom job required the data to be loaded from shapefiles and then rendered on a map based on geometry attributes. This can now be achieved in maperipy:

# first we read the shapefile...
land_use = store.load_shapefile(r"LandUse.shp")

# ... and add polygons for a specific land use type
symbol = PolygonSymbol(
    "forest", 
    land_use.find_polygons(lambda p : p["land_use_type"] == 4000))
symbol.style.fill_color = symbol.style.pen_color = Color("green")
layer.add_symbol(symbol)

Rasters

maperipy supports several types of raster grid file formats: SRTM HGT, ESRI ASCII grid and XYZ. They can now all be used to generate hillshadings or relief contours.

Example code which reads a set of XYZ files, merges them together and then generates hillshading image from them:

parts = []

dem_dir = r"DEM"

for file_name in os.listdir(dem_dir):
    grid = store.load_xyz_grid (os.path.join (dem_dir, file_name))
    parts.append(grid)

whole_dem = Raster.merge(parts)

hillshader = IgorHillshader()

hillshadingBitmap = HillShadingProcessor.shade(whole_dem, hillshader, Color("black"), 1)
hillshadingBitmap.save("dem.png")

Spatial References

One of the biggest pieces of coding I had to do was to introduce SRIDs into the picture. This allows consuming of data sources which use spatial reference systems different from the WGS 84 lon/lat used by OSM. The next step will be to actually support different SRS-es, including different coordinate systems and different map projections.

When?

This is all very nice, but when will you get to use it? Well, maperipy needs a lot more work (what I’ve shown here is just a start!) and so does the main Maperitive code. I also need to write some documentation about the API. So my guesstimate would be a couple of months. Given that a lot of new functionality has been introduced since the last “official” release, I’m thinking of naming the next major release as “Maperitive 2.0” and this will include all the stuff introduced in the September beta release.

September 21, 2011
Maperitive Beta Update

DOWNLOAD LINK: http://maperitive.net/beta/Maperitive-1.1.2001.zip

I’ve just released an update of the Maperitive beta (it’s the same download link and build number).

This update has a few nasty bugs fixed and a couple of new features.

Label Collision Detection

Maperitive now detects if two or more labels overlap. In case of overlaps the label with lower priority is removed. Priority is determined by the order of the features and rules in the rules file - rules near the beginning of the file have a higher priority than the ones near the end.

A sample rendering of UK cities and towns (the first image is from the old Maperitive and the second was generated using the latest beta):

This is a very simple system and it covers only horizontal labels (so street names can still overlap). A better system will be implemented in the future.

TileJSON

generate-tiles command now generates an accompanying TileJSON file. TileJSON is an invention of MapBox developers and it is used to hold some basic information about the generated tiles, like:

  • map name, description and attribution,
  • minimum and maximum zoom,
  • map boundaries.

The benefit of this file is that certain JavaScript mapping libraries (like Wax) can read it and automatically configure themselves based on its data. Compare that to the last step in my web map tutorial where you needed to do it manually.

Here’s a sample TileJSON file generated by Maperitive:

// Generated by Maperitive v1.1.2001
// For more information about TileJSON format, visit https://github.com/mapbox/tilejson#readme
// TODO: Update the 'tiles' to reflect your actual path on the Web server
// TODO: Update the default map location (longitude, latitude, zoom)
{
    "tilejson":"1.0.0", 
    "name":"Maperitive Web Map", 
    "description":"Maperitive Web Map", 
    "attribution":"Map data © OpenStreetMap (and) contributors, CC-BY-SA", 
    "tiles":
    [
            "tiles/{z}/{x}/{y}.png"
    ], 
    "minzoom":9, 
    "maxzoom":13, 
    "bounds":
    [
        -1.2254669867114432, 
        51.041883339771708, 
        1.5044914337393456, 
        52.076449581817052
    ], 
    "center":
    [
        0.13951222351395121, 
        51.55916646079438, 
        9
    ]
}

Note the “TODO”s: make sure you have the proper values before uploading the file to your web server.

UPDATE: there was a bug in the generated TileJSON files, “tiles” is actually an array (thanks to @kkaefer for pointing this out). I’ve updated the beta release.

September 18, 2011
New Maperitive Beta Release

DOWNLOAD LINK: http://maperitive.net/beta/Maperitive-1.1.2001.zip

A new release of Maperitive (branded as build 2001) is finally here - albeit in beta form. I wanted to release the new stuff I was working on since May, but I didn’t want to wait for everything to be totally finished. But let’s start with some…

Notes about this release you should be aware of

NOTE 1: This beta release will not appear as an update when running the “standard” Maperitive release. It will also not be possible to later auto-update it with a new release when it arrives. I’m working on a revamped auto-updating system and the old updating mechanism has been turned off in beta (that’s why it’s beta). So you will have to download the beta manually, unzip it somewhere, and run Maperitive.exe (on Windows) or Maperitive.sh (on Linux/Mac). I do not recommend overwriting the previous installation, so keep that as your backup.

NOTE 2: there have been some changes in certain Maperitive script commands, so you may need to update your scripts. The changes are mostly in how the bounds are specified when exporting bitmaps or SVGs. More on that later.

NOTE 3: there wasn’t any major bug fixing done for this release. You can see the list of open bugs (it’s not the full list, some old bugs are reported in other places - I will migrate/fix those soon).

NOTE 4: the new GUI features like map elements selection and editing may be a little bit buggy. A lot of time has been invested in making them work as there were some fundamental problems that needed to be solved. Users running Maperitive on Linux/Mac may especially notice certain problems, due to the fact that Maperitive is running through Mono. In any case, please report any problems if you see them and I will do my best to fix them.

New Features

First let me say there aren’t any revolutionary new features in this beta. Most of the work I did was done on the infrastructure code, which will help me introduce cool new things in next releases.

What is new is the way how bounds are specified (bounds specify the geographical extents of Maperitive commands). Up until now, there was only one bounds parameter, but from now on you have geometry bounds and printing bounds.

Geometry Bounds

These are more or less the “old” bounds - you use them to specify the map area to download OSM data for (as an example). Geometry bounds are defined in terms of “min longitude, min latitude, max longitude, max latitude” coordinates. They are shown as a red rectangle with diagonal hatching (see the screenshot above).

Maperitive commands that act on geometry bounds have been renamed to dump-geo-bounds, geo-bounds-use-source, reset-geo-bounds, set-geo-bounds. I haven’t updated the Maperitive documentation for these yet, but you can still use help-command to get the list of supported arguments, like:

help-commands set-geo-bounds

Printing Bounds

Printing bounds are used by export-bitmap and export-svg commands to control the exported map area. What is cool about the printing bounds is that you can specify them in terms of paper size, paper orientation, paper margins, map scale and DPI resolution. This was one of the most requested features - to be able to export a map in terms of physical paper dimensions.

Printing bounds work in two modes: fixed paper mode and free-size mode. In free-size mode you are not bound to a particular paper size, so you can define the bounds in the old way (just like geometry bounds).

Printing bounds are shown as blue dashed rectangle:

New printing bounds commands are set-print-bounds-geo, set-print-bounds-paper and set-print-bounds-view. There is also a new command for setting the paper properties: set-paper.

Why Two Boundary Types?

It may seem a bit complicated two introduce two types of boundaries, but it is necessary precondition for me to be able to introduce map rotation in Maperitive (and support for different map projections). Why? Because once you rotate the map, the geometry bounds rotate, too, so they are useless for any kind of printing boundaries (which need to be rectangles parallel with the screen).

Editing Bounds Visually

There is no need to specify bounds using the command line interface, because there are a couple of new GUI features (which gave me the greatest headache). More on that follows.

Map Context Menu

Yes, the map now has a context menu (finally). So if you want to specify printing bounds, simply right-click on the map and choose Place Printing Bounds Here. Now you have printing bounds placed on the map and selected

Selecting And Manipulating Map Elements

If you click on the geometry or printing bounds, you can select them - small yellow selection handles appear. You can drag the handles to resize the bounds. If you hold the Shift key, the resizing will maintain the aspect ratio. Hold the Control key and you will be able to resize while maintaining the center point.

Move the mouse a little bit more inside the bounds box and you will be able to move the whole bounds across the map.

Properties Window

The last major new GUI “thingy” is the Properties window. If you right-click on the printing bounds and choose Properties, a tool window will appear that lets you edit the properties. This means you don’t need to mess with command line to specify the paper size, for example.

What Comes Next

As I’ve already mentioned, Maperitive is getting a new updating system. The next “official” release will start using semantic versioning, like 2.4.2 and the old build numbering will be consigned to history. This will enable a more transparent updating system - users will be able to choose whether they want to install unstable releases or wait for the next stable/tested one.

Of course, I plan to fix some bugs too (including the ones that I’m sure you’ll report for this beta release).

Next major features: a better labeling mechanism (collision and duplicates detection, abbreviations…) and maperipy, the Python API and integrated Python scripting.

That’s really. I hope you’ll try the new beta and enjoy it. And don’t hesitate to send me your feedback (positive and/or negative)!

Liked posts on Tumblr: More liked posts »