GEE之Fusion Imagery过程记录
全部Imagery 大概720GB，总共37000多个文件，tif格式。
第一次采用分块的方法进行 fusion，主要时间是花在resources build中。
auto mask
mosaci options none
1.2495-2567
2.2568-2583
第一第二块的周边有白色的边
auto mask
mosaci options tile fill :white fill tolerance:0
3.2584-2599
4.2600-2625
5.2626-2630
6.2631-2648
全部都接边很好。如下图所示：
第二次，直接一个整体一起build，基本设置是：
auto mask
mosaci options tile fill :white fill tolerance:0resource的build过程，总共花了32个小时：
BUILD HOST: city3d02.*****.**
FUSION VERSION 5.3.0, BUILD 2019-08-27
STARTTIME: 2019-09-04 17:29:27
COMMAND: gerasterimport --imagery /gevol/lgl/assets/Resources/Imagery/GZ_IMA_001.kiasset/mosaic.kia/ver001/mosaic.khvr -o /gevol/lgl/assets/Resources/Imagery/GZ_IMA_001.kiasset/product.kia/ver001/raster.kip --north_boundary 90 --south_boundary -90 --east_boundary 180 --west_boundary -180
---------- Start Command Output ----------
Input File Sizes
2.4 MB : /gevol/lgl/assets/Resources/Imagery/GZ_IMA_001.kiasset/mosaic.kia/ver001/mosaic.khvr
2.4 MB : TOTAL Input File Sizes
Fusion Notice: Compare area of mosaic to its insets...
Fusion Notice: Loading virtual raster /gevol/lgl/assets/Resources/Imagery/GZ_IMA_001.kiasset/mosaic.kia/ver001/mosaic.khvr
Fusion Notice: Analyze virtual raster and insets areas...
Fusion Notice: Sum of inset areas: 0
Fusion Notice: Virtual raster area: 0
Fusion Notice: *.khvr file can be opened in Fusion to visually analyze virtual raster topology.
Total tiles to process: 1109065
Fusion Notice: Reprojecting source using Cubic
Fusion Notice: Approximating transform ...
Completed 0% (3931/1109065) - tiles/sec: 13.08 - time left: 23:27:54 [CPU: 100 Mem: 260 MB PF: 3]
Completed 1% (7311/1109065) - tiles/sec: 12.18 - time left: 1 Day, 01:08:13 [CPU: 99 Mem: 260 MB PF: 3]
Completed 1% (10556/1109065) - tiles/sec: 11.71 - time left: 1 Day, 02:03:14 [CPU: 99 Mem: 261 MB PF: 3]
Completed 1% (13824/1109065) - tiles/sec: 11.50 - time left: 1 Day, 02:26:39 [CPU: 99 Mem: 261 MB PF: 3]
Completed 2% (16939/1109065) - tiles/sec: 11.28 - time left: 1 Day, 02:53:55 [CPU: 99 Mem: 261 MB PF: 3]
Completed 2% (20053/1109065) - tiles/sec: 11.13 - time left: 1 Day, 03:11:05 [CPU: 99 Mem: 261 MB PF: 3]
Completed 2% (23168/1109065) - tiles/sec: 10.68 - time left: 1 Day, 04:15:18 [CPU: 99 Mem: 261 MB PF: 3]
Completed 2% (26662/1109065) - tiles/sec: 10.74 - time left: 1 Day, 04:00:12 [CPU: 99 Mem: 261 MB PF: 3]
Completed 3% (30258/1109065) - tiles/sec: 10.94 - time left: 1 Day, 03:24:11 [CPU: 99 Mem: 261 MB PF: 3]
Completed 3% (33721/1109065) - tiles/sec: 11.04 - time left: 1 Day, 03:03:03 [CPU: 99 Mem: 261 MB PF: 3]
Completed 3% (36836/1109065) - tiles/sec: 11.04 - time left: 1 Day, 02:58:59 [CPU: 99 Mem: 261 MB PF: 3]
Completed 4% (39934/1109065) - tiles/sec: 11.03 - time left: 1 Day, 02:55:30 [CPU: 99 Mem: 261 MB PF: 3]
Completed 4% (43048/1109065) - tiles/sec: 11.03 - time left: 1 Day, 02:51:28 [CPU: 99 Mem: 261 MB PF: 3]
Completed 4% (46155/1109065) - tiles/sec: 10.81 - time left: 1 Day, 03:19:01 [CPU: 99 Mem: 261 MB PF: 3]
Completed 4% (49265/1109065) - tiles/sec: 10.54 - time left: 1 Day, 03:55:58 [CPU: 99 Mem: 261 MB PF: 3]
Completed 5% (52374/1109065) - tiles/sec: 10.34 - time left: 1 Day, 04:23:04 [CPU: 99 Mem: 261 MB PF: 3]
Completed 5% (55485/1109065) - tiles/sec: 10.34 - time left: 1 Day, 04:18:15 [CPU: 99 Mem: 261 MB PF: 3]
Completed 5% (58927/1109065) - tiles/sec: 10.53 - time left: 1 Day, 03:41:60 [CPU: 99 Mem: 262 MB PF: 3]
Completed 6% (62250/1109065) - tiles/sec: 10.65 - time left: 1 Day, 03:18:31 [CPU: 99 Mem: 262 MB PF: 3]
Completed 6% (65728/1109065) - tiles/sec: 10.85 - time left: 1 Day, 02:42:09 [CPU: 99 Mem: 262 MB PF: 3]
Completed 6% (68979/1109065) - tiles/sec: 10.93 - time left: 1 Day, 02:25:56 [CPU: 99 Mem: 262 MB PF: 3]
Completed 6% (72085/1109065) - tiles/sec: 10.93 - time left: 1 Day, 02:21:27 [CPU: 99 Mem: 262 MB PF: 3]
Completed 7% (75194/1109065) - tiles/sec: 10.93 - time left: 1 Day, 02:16:53 [CPU: 99 Mem: 262 MB PF: 3]
Completed 7% (78298/1109065) - tiles/sec: 10.74 - time left: 1 Day, 02:39:57 [CPU: 99 Mem: 262 MB PF: 3]
Completed 7% (81401/1109065) - tiles/sec: 10.62 - time left: 1 Day, 02:53:03 [CPU: 99 Mem: 262 MB PF: 3]
Completed 8% (84501/1109065) - tiles/sec: 10.41 - time left: 1 Day, 03:20:01 [CPU: 99 Mem: 262 MB PF: 3]
Completed 8% (87611/1109065) - tiles/sec: 10.33 - time left: 1 Day, 03:27:58 [CPU: 99 Mem: 262 MB PF: 3]
Completed 8% (90920/1109065) - tiles/sec: 10.45 - time left: 1 Day, 03:04:26 [CPU: 99 Mem: 262 MB PF: 3]
Completed 9% (94281/1109065) - tiles/sec: 10.59 - time left: 1 Day, 02:37:11 [CPU: 99 Mem: 262 MB PF: 3]
*******省略**********
Completed 62% (686967/1109065) - tiles/sec: 8.27 - time left: 14:10:28 [CPU: 97 Mem: 1600 MB PF: 4]
Completed 62% (689451/1109065) - tiles/sec: 8.27 - time left: 14:05:20 [CPU: 97 Mem: 1598 MB PF: 4]
Completed 62% (691942/1109065) - tiles/sec: 8.27 - time left: 14:01:04 [CPU: 97 Mem: 1690 MB PF: 4]
Completed 63% (694419/1109065) - tiles/sec: 8.26 - time left: 13:57:05 [CPU: 97 Mem: 1658 MB PF: 4]
Completed 63% (696937/1109065) - tiles/sec: 8.27 - time left: 13:50:05 [CPU: 97 Mem: 1609 MB PF: 4]
Completed 63% (699522/1109065) - tiles/sec: 8.35 - time left: 13:37:17 [CPU: 97 Mem: 1817 MB PF: 4]
Completed 63% (702131/1109065) - tiles/sec: 8.41 - time left: 13:26:48 [CPU: 97 Mem: 1802 MB PF: 4]
Completed 64% (704597/1109065) - tiles/sec: 8.40 - time left: 13:22:48 [CPU: 97 Mem: 1763 MB PF: 4]
Completed 64% (707069/1109065) - tiles/sec: 8.39 - time left: 13:18:40 [CPU: 97 Mem: 1763 MB PF: 4]
******省略************
Completed 82% (908361/1109065) - tiles/sec: 9.33 - time left: 05:58:31 [CPU: 97 Mem: 1644 MB PF: 4]
Completed 82% (910933/1109065) - tiles/sec: 9.35 - time left: 05:53:02 [CPU: 97 Mem: 1562 MB PF: 4]
Completed 82% (913412/1109065) - tiles/sec: 9.37 - time left: 05:48:00 [CPU: 97 Mem: 1562 MB PF: 4]
Completed 83% (915848/1109065) - tiles/sec: 9.20 - time left: 05:49:56 [CPU: 97 Mem: 1562 MB PF: 4]
Completed 83% (918308/1109065) - tiles/sec: 8.82 - time left: 06:00:20 [CPU: 97 Mem: 1552 MB PF: 4]
Completed 83% (920751/1109065) - tiles/sec: 8.53 - time left: 06:08:04 [CPU: 97 Mem: 1524 MB PF: 4]
Completed 83% (923310/1109065) - tiles/sec: 8.29 - time left: 06:13:28 [CPU: 97 Mem: 1524 MB PF: 4]
Completed 83% (925855/1109065) - tiles/sec: 8.28 - time left: 06:08:58 [CPU: 97 Mem: 1631 MB PF: 4]
Completed 84% (928308/1109065) - tiles/sec: 8.26 - time left: 06:04:36 [CPU: 97 Mem: 1607 MB PF: 4]
Completed 84% (930873/1109065) - tiles/sec: 8.33 - time left: 05:56:25 [CPU: 97 Mem: 1657 MB PF: 4]
Completed 84% (934336/1109065) - tiles/sec: 8.89 - time left: 05:27:30 [CPU: 97 Mem: 1657 MB PF: 4]
Completed 85% (937586/1109065) - tiles/sec: 9.34 - time left: 05:05:59 [CPU: 97 Mem: 1657 MB PF: 4]
Completed 85% (940720/1109065) - tiles/sec: 9.66 - time left: 04:50:32 [CPU: 97 Mem: 1657 MB PF: 4]
Completed 85% (943912/1109065) - tiles/sec: 10.01 - time left: 04:34:52 [CPU: 97 Mem: 1657 MB PF: 4]
Completed 85% (947101/1109065) - tiles/sec: 10.42 - time left: 04:18:58 [CPU: 97 Mem: 1657 MB PF: 4]
Completed 86% (950239/1109065) - tiles/sec: 10.74 - time left: 04:06:25 [CPU: 97 Mem: 1657 MB PF: 4]
Completed 86% (953425/1109065) - tiles/sec: 10.58 - time left: 04:05:05 [CPU: 97 Mem: 1657 MB PF: 4]
Completed 86% (956609/1109065) - tiles/sec: 10.54 - time left: 04:00:58 [CPU: 97 Mem: 1657 MB PF: 4]
Completed 87% (959754/1109065) - tiles/sec: 10.55 - time left: 03:55:56 [CPU: 97 Mem: 1657 MB PF: 4]
Completed 87% (962933/1109065) - tiles/sec: 10.54 - time left: 03:51:05 [CPU: 97 Mem: 1657 MB PF: 4]
Completed 87% (966118/1109065) - tiles/sec: 10.54 - time left: 03:46:07 [CPU: 97 Mem: 1657 MB PF: 4]
Completed 87% (969260/1109065) - tiles/sec: 10.54 - time left: 03:41:10 [CPU: 97 Mem: 1657 MB PF: 4]
Completed 88% (972441/1109065) - tiles/sec: 10.53 - time left: 03:36:10 [CPU: 97 Mem: 1657 MB PF: 4]
Completed 97% (1079557/1109065) - tiles/sec: 8.78 - time left: 00:56:03 [CPU: 97 Mem: 1760 MB PF: 4]
Completed 98% (1083093/1109065) - tiles/sec: 9.22 - time left: 00:46:58 [CPU: 97 Mem: 1760 MB PF: 4]
Completed 98% (1086509/1109065) - tiles/sec: 9.63 - time left: 00:39:03 [CPU: 97 Mem: 1760 MB PF: 4]
Completed 98% (1089926/1109065) - tiles/sec: 10.16 - time left: 00:31:24 [CPU: 97 Mem: 1760 MB PF: 4]
Completed 99% (1093297/1109065) - tiles/sec: 10.69 - time left: 00:24:35 [CPU: 97 Mem: 1760 MB PF: 4]
Completed 99% (1096623/1109065) - tiles/sec: 11.11 - time left: 00:18:40 [CPU: 97 Mem: 1760 MB PF: 4]
Completed 99% (1099903/1109065) - tiles/sec: 11.29 - time left: 00:13:32 [CPU: 97 Mem: 1760 MB PF: 4]
Completed 99% (1103007/1109065) - tiles/sec: 11.05 - time left: 00:09:08 [CPU: 97 Mem: 1760 MB PF: 4]
Completed 100% (1105710/1109065) - tiles/sec: 10.65 - time left: 00:05:15 [CPU: 97 Mem: 1760 MB PF: 4]
Completed 100% (1108501/1109065) - tiles/sec: 10.31 - time left: 00:00:55 [CPU: 97 Mem: 1756 MB PF: 4]
Completed 100% (1109065/1109065) - tiles/sec: 10.17 - time left: 00:00:00 [CPU: 97 Mem: 1690 MB PF: 4]
Processed 1109065 tiles
Total time to process: 1 Day, 08:39:31
Average tiles per second: 9.43
Output File Sizes
362.73 GB : /gevol/lgl/assets/Resources/Imagery/GZ_IMA_001.kiasset/product.kia/ver001/raster.kip
362.73 GB : TOTAL Output File Sizes
----------------------------- ----------
Process Stats
117571.81 real time
114195.16 user time
595.35 system time
97% CPU usage average
0 swaps
4 page faults
729443 I/O context switches
Peak memory used for this job: 1875360 kB
---------- End Command Output ----------
ENDTIME: 2019-09-06 02:08:59
ELAPSEDTIME: 117572 seconds (32h39m32s)
COMPLETED SUCCESSFULLY
COMMAND: gemaskgen --oformat=PNG --preview --maxsize=1024 /gevol/lgl/assets/Resources/Imagery/GZ_IMA_001.kiasset/product.kia/ver001/raster.kip /gevol/lgl/assets/Resources/Imagery/GZ_IMA_001.kiasset/product.kia/ver001/preview.png
---------- Start Command Output ----------
Input File Sizes
362.73 GB : /gevol/lgl/assets/Resources/Imagery/GZ_IMA_001.kiasset/product.kia/ver001/raster.kip
362.73 GB : TOTAL Input File Sizes
Fusion Notice: raster type: Imagery
Fusion Notice: num components: 3
Fusion Notice: component size: 1
Fusion Notice: component type: UInt8
Fusion Notice: min, max product levels: 8, 28
Fusion Notice: image size at max product level, pixels: (wh) 835287, 1039694
Fusion Notice: Mask parameters:
Fusion Notice: band selection: 1
Fusion Notice: automatic level selection for generating mask: 18
Fusion Notice: output image offset, pixels: (xy) 320, 36
Fusion Notice: output image size, pixels: (wh) 816, 1015
Fusion Notice: adjusted feather size: 0
Fusion Notice: adjusted hole size: 0
Fusion Notice: Flood filling image in memory
Flooded 1 tiles (500.0/sec) in 00:00:00
Fusion Notice: Writing imagery preview /gevol/lgl/assets/Resources/Imagery/GZ_IMA_001.kiasset/product.kia/ver001/preview.png
Output File Sizes
944.9 KB : /gevol/lgl/assets/Resources/Imagery/GZ_IMA_001.kiasset/product.kia/ver001/preview.png
944.9 KB : TOTAL Output File Sizes
----------------------------- ----------
Process Stats
0.48 real time
0.36 user time
0.05 system time
86% CPU usage average
0 swaps
79 page faults
116 I/O context switches
Peak memory used for this job: 222928 kB
---------- End Command Output ----------
ENDTIME: 2019-09-06 02:08:59
ELAPSEDTIME: 0 seconds
COMPLETED SUCCESSFULLY
TOTAL ELAPSEDTIME: 117572 seconds (32h39m32s)