Dstile howto

from http://worldwindcentral.com/wiki/Dstile_howto

This is a step-by-step tutorial on how to get your own georeferenced imagery into World Wind. This tutorial will show you how to create tiles from a georeferenced image then create a layer for the tiles in World Wind. From there you can package up the layer as an add-on to distribute. This layer will be stored entirely on your computer. If you want to serve the tiles to others then this tutorial may not be helpful.

The organization of World Wind's tile system is summarized here. This tutorial assumes that you already understand how the tile system works and how World Wind's xml files work.

I will be walking you through an example layer that I will create. The steps can be easily modified for your own data.

[edit]Required Software

• FWTools - This is a free package of tools for manipulating georeferenced imagery. It is required for dstile to work. The entire package is an 18MB download and 84.5MB installed. Perhaps in the future someone can package up only what is required for dstile to work. You can save some space by not installing the developer files using the options in the installer. dstile may work with the newer 2.x versions of FWTools, but it was developed against the 1.x series.
• The dstile executable itself - This link is the most recent version (13-08-2007). The general procedure that this program follows is summarized in the Making Layersarticle. Put dstile.exe in the folder where FWTools was installed (e.g. C:\Program Files\FWTools1.2.0\bin).
• World Wind, obviously

[edit]Preparation

In addition to the software you will need a georeferenced image. For this tutorial, I will be using a geology map of northern Delaware from here. The format of the imagery can be anything supported by FWTools (complete list here). Dstile can not handle indexed images at the moment, but these can be converted to RGB Images using the pct2rgb utility suppiled with FWTools.

I put the image and world file (if the image requires one) in the FWTools directory for simplicity, but these files can be anywhere as long as you give dstile the correct location for them.

Dstile can also tile terrain data using the --dem switch at the command line. The terrain data must be 16bit.

[edit]Creating tiles

Dstile is a command line program (though a gui is on the way). Run the "FWTools shell" shortcut from your start menu. This will give you a command prompt window in the FWTools directory. Alternately you can start your own command prompt, navigate to the FWTools directory, and run the "setfw.bat" script.

The general command line to create local tiles is as follows:

dstile.exe tile --lztsd tile_size --wwcache --overviews output_directory georeferenced_image_file(s)

The variables are as follows:

• tile_size - the size of the coarsest level of tiles, in degrees (more on this below)
• output_directory - the directory where the tile system will be placed as it is created
• georeferenced_image_file(s) - the image file itself. A list of multiple image files can be supplied and will be automatically combined into a single set of tiles. Alternately, gdal_merge.py in FWTools can be used to create your own mosaic first if you would like more control over that step of the process.

The command line switches:

• --tile tells dstile that you want to create tiles.
• --wwcache tells dstile to organize and name the tiles in the way that World Wind expects them to appear.
• --overviews tells dstile to create all of the tile levels necessary. Leaving out this option will result in only the most detailed level of tiles being created.
• --png is used to create .png format tiles (slow but lossless compression)
• --float is used to tile floating point dem data. ( can be slightly more accurate than Int16)
• --srcProj is used to specify projection for files without one e.g .tif/.tfw set, use the epsg code, e.g. epsg:32754 is UTM Zone 54S , epsg:4326 is WGS84.

For my example geology map, the command line I will use to generate the tiles is as follows:

dstile.exe tile --lztsd 1.0 --wwcache --overviews output_tiles geomap10.tif

This will create a new directory (in my FWTools directory) called "output_tiles" where it will put the tile system. The georeferenced input file is "geomap10.tif". Note that this input file needs to be the image itself, not the world file (geomap10.tfw in this case), though the world file does need to exist in the same folder as the image. In my example it created six levels of tiles (folders name 0 through 5) in the output_tiles directory.

When dstile runs, it will restate some of the input parameters and then show a progress indicator. The processing took just a minute or two for my 111MB image file. In another case, dstile running on a single machine took 70 minutes to produce a tile tree of 1.8GB from a 21GB data set. Larger runs are done on the Free Earth Foundation Cluster.

[edit]Level zero tile size

This parameter controls at what zoom level or camera altitude your imagery layer will appear. A larger level zero tile size means it will appear sooner as you zoom in, a smaller size means it will appear later as you zoom in. For reference, the blue marble uses a level zero tile size of 36 degrees, World Wind's default landsat layer uses a level zero tile size of 2.25 degrees, and the high resolution imagery in the Zoomit! add-on uses level zero tile sizes as small as 0.1 to 0.2 degrees.

For my example layer with the geologic map, I used a level zero tile size of 1.0 degrees. This causes the layer to appear at an altitude of around 390 km, as seen in this screenshot. You will have to experiment with this parameter to make your layer work the way you want it to.

The level zero tile size should be 180/(2^n), where n is an integer. That is, a lztsd of 1.40625 (180/(2^7)) would have been the ideal value to use for this example. It is possible to give dstile values that do not satisfy this expression, and the resulting imagery layers will render in WW, but there may be alignment problems with the tiles. The map in this example did not show the alignment problems because of the relatively coarse scale. Detailed orthoimagery will be more likely to show the alignment problems, and thus the 180/(2^n) rule should be followed.

[edit]Adding the layer to World Wind

To make the layer appear in World Wind you need to put the tiles somewhere in your World Wind directory and then create an xml file for the new layer.

For this example, I put the tiles in my \Data\Earth\ directory. Alternately, they could go somewhere in the /Add-ons/ directory. I don't like putting my own data into the \Cache\ directory because I occasionally delete everything in \Cache\ if I get low on disk space. I like to reserve \Cache\ for data that can be re-downloaded if necessary. Thus I moved the contents of the "output_tile" directory (the contents should only consist of the numbered folders for each level) that dstile created into "\Data\Earth\N. Delaware geology map\" in my World Wind directory.

The organization of these files in the World Wind directory is up to you. If I were going to be putting a lot of geology maps into World Wind then I might want to add another folder and put the tiles into it (\Data\Earth\Geology maps\N. Delaware\ for example). As long as you tell World Wind where to find them you can organize them however you want.

Once the tiles are in the right location, we need an xml file to create the layer in World Wind. The xml files goes in \Config\Earth\ in the World Wind directory. It is helpful to name the xml file to match the name of the layer it is creating ("geology maps.xml" in this case). The xml I used is below:

<?xml version="1.0" encoding="UTF-8"?>
<LayerSet Name="Geology maps" ShowOnlyOneLayer="false" ShowAtStartup="false" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="LayerSet.xsd">
	<QuadTileSet ShowAtStartup="true">
		<Name>N. Delaware</Name>
		<DistanceAboveSurface>0</DistanceAboveSurface>
		<BoundingBox>
			<North>
				<Value>40.0</Value>
			</North>
			<South>
				<Value>39.0</Value>
			</South>
			<West>
				<Value>-76.0</Value>
			</West>
			<East>
				<Value>-75.0</Value>
			</East>
		</BoundingBox>
		<TerrainMapped>true</TerrainMapped>
		<ImageAccessor>
			<LevelZeroTileSizeDegrees>1.0</LevelZeroTileSizeDegrees>
			<NumberLevels>6</NumberLevels>
			<TextureSizePixels>512</TextureSizePixels>
			<ImageFileExtension>jpg</ImageFileExtension>
			<PermanentDirectory>Data\Earth\N. Delaware geology map</PermanentDirectory>
		</ImageAccessor>
		<TransparentColor><Red>0</Red><Green>0</Green><Blue>0</Blue></TransparentColor>
	</QuadTileSet>
</LayerSet>

The "LevelZeroTileSizeDegrees" field needs to be the size that was used in dstile and the "PermanentDirectory" field needs to be the directory where you put the tiles. This will create an entry in the layer manager called "Geology maps" and inside of it there will be a sublayer called "N. Delaware".

Dstile fills in the empty parts of the tiles outside of your imagery with black, so the "TransparentColor" line in the xml makes this area transparent. Note that this will also make solid black areas inside of your imagery transparent. A few preprocessing steps can be used to fix this, in general a bit of creativity is needed. For example in an image with default null value and assuming no pixels within the actual image are pure white, a quick fix will be to:

• Add an offset (e.g. 32) to all black pixels.
• Convert all white pixels to black
• Tile using dstile and set the transparent colour as above.

Due to compression in JPEG's colour along black edges is not pure black. To get around this use the following tags:

    <TransparentMinValue>0</TransparentMinValue>
    <TransparentMaxValue>20</TransparentMaxValue>

Experiment with the TransparentMaxValue to get the best visual results, 20 is only a suggestion.

[edit]Custom DEMs

The procedure for adding custom DEMs into WorldWind is very similar to imagery; however, the tiles and the xml code is added to the cache and earth.xml respectively, instead of the Data folder and a custom xml.

[edit]Dstile command

The DEM tif must be in float32 or int16 format and 1 band only (greyscale). The command is:

 dstile tile --lztsd <level zero tile size in degrees> --wwcache [tile out in worldwind cache format] --dem [use dem tiling switch] [--srcProj epsg:4326] --overviews [build higher level overviews] <destination directory> <sourcefile(s)>

...which is similar to above, but with the "--dem" flag added.

[edit]Tile location

Put the tiles in a custom folder in the SRTM cache folder, say X:\WorldWind\Cache\Earth\SRTM\<Custom DEM>.

[edit]XML layout

In Earth.xml under the config folder, add a Higher Resolution Subset under the TerrainAccessor tag, with the name and datasetname nodes being the same name you gave the folder in the cache, and a format like so:

   <HigherResolutionSubsets Name="Custom DEM">
     <TerrainTileService>
       <ServerUrl>http://127.0.0.1</ServerUrl>
       <DataSetName>Custom DEM</DataSetName>
       <LevelZeroTileSizeDegrees>0.5</LevelZeroTileSizeDegrees>
       <NumberLevels>6</NumberLevels>
       <SamplesPerTile>512</SamplesPerTile>
       <DataFormat>Int16</DataFormat>
       <FileExtension>bil</FileExtension>
     </TerrainTileService>
     <LatLonBoundingBox>
       <North>
         <Value>0.0</Value>
       </North>
       <South>
         <Value>-1.0</Value>
       </South>
       <West>
         <Value>0.0</Value>
       </West>
       <East>
         <Value>1.0</Value>
       </East>
     </LatLonBoundingBox>
   </HigherResolutionSubsets>

And, voila! Start WorldWind. By the way, if you are using the version from the SVN, you are able to use the Terrain Manager to turn this layer on and off; caution, you may need to zoom out and back in for it to switch, and need to wait for a minute if you do not have a moderately powerful machine (like over an AMD 3800, 2GB RAM, Geforce 6600GT)...

[edit]Tips

• When experimenting with the level zero tile size, it may be a good idea to erase the contents of the output folder before rerunning dstile. Dstile will merge new tiles with existing ones where they overlap, but will not erase any old tiles left over that aren't used.
• It may be more useful to have dstile output the tiles directly where you want them in your World Wind directory. I used a temporary directory in this example to make it simpler to follow. If you want to put the tiles directly into the World Wind directory you could use:

dstile.exe tile --lztsd 1.0 --wwcache --overviews "C:\Program Files\NASA\World Wind 1.4\Add-ons\Earth\geomap10" geomap10.tif

• In order to be able to tile the image you have to make sure the data does not extend beyond the maxiumum range of the WGS84 datum (-180, -90, 180, 90). To find out if this is the case with your data, simply type in:

gdalinfo geomap10.tif 

If the values displayed for ul ll ur lr go beyond these limits, a clip using "gdal_translate -projwin" might come in handy. Beware, -projwin requires ulx uly lrx lry:

minx = upper left x
miny = lower right y
maxx = lower right x
maxy = upper left y

You can now do a clip using:

gdal_translate -projwin minx maxy maxx miny input.tif output.tif

I have noticed that providing whole rounded off numbers sometimes still had the data extend beyond the clip range entered, in that case I worked around the problem by using decimals and reducing the extent slightly. For example for a global file (NOTE: not for the geomap10.tif mentioned earlier) DSTile worked when I clipped it using:

gdal_translate.exe -projwin -179.987 89.985 179.988 -89.98 input.tif output.tif

Also thanks to Nowak there is a driver for GDAL that does the inverse of DSTile. You can point it at at various WMS style TileCaches and produce a single file.

Here is an example config file for the WorldWind servers:

 <GDAL_WMS>
   <Service name="WorldWind">
       <ServerUrl>http://.../...?</ServerUrl>
       <Layer>...</Layer>
   </Service>
   <DataWindow>
       <UpperLeftX>-180.0</UpperLeftX>
       <UpperLeftY>90.0</UpperLeftY>
       <LowerRightX>180.0</LowerRightX>
       <LowerRightY>-90.0</LowerRightY>
       <TileLevel>level count - 1</TileLevel>
       <TileCountX>360 / lztsd</TileCountX>
       <TileCountY>180 / lztsd</TileCountY>
   </DataWindow>
    <BandsCount>3</BandsCount>
   <BlockSizeX>512</BlockSizeX>
   <BlockSizeY>512</BlockSizeY>
 </GDAL_WMS>

[edit]DSTile Credits

• Adam Nowacki: For writing this amazingly simple piece of GDAL wizardry.
• Frank Warmerdam: For maintaining the GDAL library and FWTools.
• Tisham Dhar: For wasting his time in porting to the humble and buggy Windows platform.
• Erik Newman: For writing this HOW-TO.
• Mark de Blois: For adding extra information to this HOW-TO.