GeoTools : 3D Rendering Pipeline Status

2007-09-28 Tiled Map Renderer

The FOSS4G conference has been great so far, and has given me a lot of ideas for how to continue the 3D renderer, as well as an idea of what other open source solutions exists within the same area. I'll get back to those in more depth after the conference.

For now, on the right is a screenshot of a tiled map, using multi-detail level satellite image tiles. The tiles are generated by GDAL2Tiles, the project of Petr Klokan, another of the summer of code programmers. The implementation is a relatively quick hack for now, just to demonstrate that loading tiles from disk is also a possible way to create the ground raster.

2007-09-21 Going to FOSS4G

I'm going to be present on the FOSS4G GIS conference in Victoria, Canada, to demonstrate the 3D Map Renderer. I'll be in the Google booth, along with other summer of code students.

I created a short powerpoint presentation of the 3D Map Renderer, for handouts and such.

The performance of the 3D renderer is really not that good with large maps, as the used 2D map renderer is not good at efficiently rendering only a small part of a map at a time. Maybe I'll take a stab at implementing a custom random map generator on the plane, just to demonstrate the ideal speed of the renderer (wink)...

Another problem with the current approach of rendering a textured version of the 2D map on the ground plane of the 3D map is that it requires a lot of memory. It also doesn't give a very realistic end result.

I'm planning to experiment with using pre-defined tiled, terrain type specific textures (e.g. different ones for woods, suburbs, cities, fields, etc). Roads, borders, lakes, and other similar features would then have to be presented as 3D objects, as they can no longer be rendered on the ground. This approach could look better and would be faster, at least as long as the 3D objects don't get too intricate.

2007-08-25 It's ready!

Today I finally fixed the last major bug, and switched to a larger dataset. It's now ready for review.

In the end, some major features like elevation are left to do, as well as some of the more fancy planned features like 3D markers. Also, the resolution is not that high in the current map, due to memory constraints - increasing the amount of memory for the JVM will allow using higher resolutions though. The rendering order of tiles could be optimized, to give it a more responsive feel (now it starts off by rendering all the large tiles that are invisible, so the user sees no change for a while after starting up). Also, it seems the size of the map affects the rendering performance of the 2D StreamingRenderer used, even when a tile doesn't contain any actual features. This will make using the 3D renderer for large maps unpractical, until the StreamingRenderer is optimized.

But all in all, it works, it's a relatively solid foundation for future work, and it was a challenging and fun summer project to do. Thanks to Jody for mentoring me, GeoTools for accepting my application, and Google for the Summer of Code program!

2007-08-18 Bugs killed, now to apply some polish

I had some hard-to-track-down bugs involving the terrain blocks being layered on top of each other, and the textures not being updated when terrain blocks were re-used. Now they are both fixed. My laptop also shows a pretty consistent 90 FPS when moving around the terrain (I capped the max rendering speed to that IIRC), so it looks like I don't have to be concerned about performance issues.

Recently I also implemented rendering of the map texture in a separate thread, so it doesn't slow down rendering.

Next I'll have terrain blocks that still are rendering their texture re-use a part of their parent texture instead.
Another outstanding thing that needs to be fixed is to enforce that no quad tree node can differ more than one subdivision level from a neighbour. If I have time after that tomorrow I might take a stab at implementing elevation support (it will be retrieved in basically the same way as the textures, so I have the pattern designed already).

2007-08-12 Fixed library path to be cross platform

By putting the native libraries in the same directory it is no longer necessary to use a different java.library.path on different environments, so now the demo should be runnable with mvn exec:exec on linux and osx also without editing the pom.xml.

2007-08-11 Fixing memory leaks

I implemented pooling of the BufferedImages that the ground is rendered to. That seemed to be the main cause of the memory problems.

I still plan to implement pooling of the TerrainBlock objects too, so that unused 3D objects are just repositioned and get new elevation and texture, instead of being created from scratch.

Status 2007-08-07

Implemented block management over the weekend, now blocks are split into smaller ones when the camera gets closer, and new larger blocks are created when the camera gets closer to the edge of the current quad tree.

There are still some tuning to do with the splitting, I'll probably need to enforce a constraint that neighboring blocks should differ from each other by at most one subdivision. This is also needed for creating smooth seams between blocks when I add the elevation data.

There was some problems with memory running out fast, when allocating textures. I'll need to investigate if the textures uploaded to the 3D card need to be released (this is probably the cause). If also the JVM environment is running out of memory due to ground texture images being created then I'll need to check how much heap space is reserved for the JVM, and recommend increasing it. There might also be some memory leaks if the garbage collector can not find the unused ground images of unused blocks for some reason. In addition, the renderer should handle running out of texture memory gracefully (use a placeholder texture instead).

After some investigation of the memory issues, I'll move on to implementing the elevated terrain.

I'll commit the changes when I get home later today.

Added some screenshots to the wiki page also.

Status 2007-08-01 Update

Made the test program a bit clearer by labeling the views, and changing the terrain block back to use random colors until a texture is available.

Also added a rotate gesture, and changed the pan to pan along camera up and side vectors. Now it's possible to look at the 3D map from different directions.

Status 2007-08-01

Fixed the maven build process, by adding the required non-mavenized third party libraries and native libraries to the svn repository, and writing an install script that adds them to the local maven repository.

The install_third_party_libraries script should be run once before any builds, then mvn clean install can be used to build the 3d renderer, and mvn exec:exec can be used to run the test program.

Status 2007-07-29

Sorry for the long absence, I got distracted with learning the Scala programming language (http://www.scala-lang.org/) last week.

Now I'm back though; today I did some refactoring and implementation of the quad tree code for the 3D terrain: I extracted the bounding box related code from the QuadTreeNode to a separate class (BoundingRectangle) and refactored it, this simplified the quad tree considerably. I also decided to store the center coordinates of the bounding rectangle, this avoids errors that could happen because of numerical innaccuracies when calculating the same coordinate through different steps.

The pom.xml file is also updated to java 5, although I still need to come up with some way to include the non mavenized libraries in the build.

Next I'm planning to define interfaces for passing texture and height data to the renderer, add some unit tests for the quad tree, and start working on map blocks.

(I'm having trouble connecting to the geotools svn from my laptop, so I'll retry delivering the code tomorrow)

Status 2007-07-12

I'm on vacation until next week, 17 July, so no progress this week.

Status 2007-07-04

Brainstormed a little around the navigation (thanks Demitar! (smile)), and came up with a pretty satisfactory solution. Now a normal three button mouse with a scrollwheel should be enough for effortless navigation around the map. The 'reset to nearest standard view' on middle button click in particular should be useful - no need to carefully align the camera to a specific axis, or hunt for the right alignment button.

Status 2007-07-03 update

Implemented the skirt along the edges of a TerrainMesh, as well as a way of setting the texture of a TerrainMesh directly from an image.

Texture management may need some attention later, texture memory space is a relatively scarce resource, and we need to make sure unused textures are released.

Status 2007-07-03

Implemented the TerrainMesh, now we have one terrain block containing a terrain mesh consisting of 32 * 32 * 2 (exact numbers will be tunable) randomly colored triangles in the test application.

Also discovered a new issue: Lightweight Swing components do not render on top of the 3D Canvas (see issue list below for full description). It's not a major showstopper, but can be annoying. Requires a bit of research to see if there are any easy workarounds.

Next step is to add a downturned 'skirt' along the outer edges of each TerrainMesh, so that one pixel cracks due to rounding imprecisions are not visible between adjacent TerrainMeshes. There is already some code for this in the repository. After that I'll continue with texturing the TerrainMeshes, and storing height data into them.

Status 2007-07-02

I moved the task lists to a wiki list instead of the dynamic list, as the dynamic list was bugging (forgetting changes, and resetting the status information).

Status 2007-07-01

Implemented a framework for navigation gestures, and a simple panning gesture.

Also refactored the code, so that the Canvas3D is a separate (reusable) class, and the Renderer3D can focus on managing the interface between the 3D terrain model and the MapContext.

In addition, I cleared up the package structure a bit.

Status 2007-06-28

First terrain block showing up in the 3D view in the test application. For now it is just a colored box created by the TerrainBlockImpl class, later I'll replace it with a real textured mesh.

To enable better viewing of the 3D output some kind of 3D navigation is needed. I'm thinking of mouse wheel zoom, panning with one mouse button, and turning the ground with the other. In addition there could be free flight like navigation with WASD and arrow keys. This is probably something that clients want to configure themselves in more detail, so I'll create some interface for the navigation, with one or two default implementations.

Status 2007-06-27

According to Jody there are a few Java 1.5 modules already for GeoTools, so it would be ok to use Java 1.5 also. That should solve the issue from the previous entry, and also use of generics and other nice 1.5 features, so I think I'll make the 3D renderer a Java 1.5 module.

Status 2007-06-26

Currently I'm working on the map block class, rendering a part
of a 2D map to a texture (using the existing Java2D renderer), and
displaying that texture on a terrain mesh covering the corresponding
ground area in 3D, with altitude information from some coverage.

The next step will be management of the map blocks - creating them around
the camera, subdividing them into smaller map blocks when the camera
moves closer, and merging them back into a larger map block when the
camera moves away (will probably be done by just hiding the larger
block, and creating / showing the smaller blocks, and vice versa).

After that I can work on 3D markers, loading 3D models or 2D icons and
displaying them at locations specified by points.

If I still have time after that, I can look into some of the other
things I described in the design doc, like forest rendering.

Status 2007-06-13

I imported some quadtree code I wrote earlier, planning to use that for the 3D terrain blocks making up the 3D view.

I was thinking of making a simple test data generator (that way I can easily create the kind of features I need to test, and can create performance test material easy too. Plus it's a way to familiarize myself with the feature structures).

As jgarnett pointed out though, real data is more messy, and will thus easier expose various bugs in the implementation, so real data should also be used regularily when testing the 3D renderer.

I have a test application that opens up a 3D view (just a stub at the moment) and a 2D view of the same map area.

Towards the end of the week (15-17.6.2007 or so) I should probably start to work on the actual 3D code.

I'll upload my current code into my spike directory asap for review.