Haoran Wu
In the last episode I touched upon the usage of map
function in GEE. Now we can easily perform any mathematical operations on objects of ee.Image
or ee.ImageCollection
classes. For advanced analyses, we can also simply google it can get codes from different sources. The final annoying thing about basic operations in GEE, however, is the error messages in our screen, or unexpected results we get from our code.
In this post, I would like to share some techniques on dealing with error messages and unexpected results. To put it more exactly, I would like to introduce:
The second problem is more complicated, but unfortunately quite common for advanced users.
For managing error messages, I suggest two ways:
print
function. GEE Code Editor is neither RStudio nor Spyder, which means that we could not executive one command and see the result intermediately. Instead, what we often get is the whole results from a wordy GEE script. Sometimes we are confused about (1) which steps triggers the error messages; (2) why an GEE function cannot be applied to a variable of a certain type (where the error messages usually go like “XXX is not a function”). In these cases, print detailed information about a variable is recommended.print("var = ", var)
imgColl.map(function(img){
// ...
return img.add(10);
})
Since we cannot use the print function within loops, an alternative way is to run the following code for debugging,
var img = imgColl.first();
// ... You can also add the `print` function here
var result = img.add(10);
print(result);
In remote sensing, we often need a bundle of steps for pre-processing, which means that an incorrect results without error messages in any steps is disastrous. In this sense, it is important to use charts to check our results.
For instance, a scatter plot can be created using the following code:
var point = ee.Geometry.Point([-80.498, 37.254]);
var ndvi = ee.ImageCollection("NASA/GIMMS/3GV0");
var chart = ui.Chart.image.series(ndvi.select(['ndvi']), point)
.setSeriesNames(['NDVI'])
.setChartType('ScatterChart')
.setOptions({
title: 'NDVI Time Series',
trendlines: {
0: {color: 'CC0000'}
},
lineWidth: 1,
pointSize: 3,
});
print(chart);
For a histogram, we can use:
var NorthAmerica =
ee.Geometry.Polygon(
[[[-150, 60],
[-150, 40],
[-51, 40],
[-51, 60]]], null, false);
var ndvi_img = ee.ImageCollection("MODIS/061/MOD13A1").first();
var chart =
ui.Chart.image.histogram({image: ndvi_img.select("NDVI"), region: NorthAmerica, scale: 500, maxPixels: 1e13, maxBuckets: 100})
.setSeriesNames(['NDVI'])
.setOptions({
title: 'NDVI Histogram',
hAxis: {
title: 'Reflectance (scaled by 1e4)',
titleTextStyle: {italic: false, bold: true},
},
vAxis:
{title: 'Count', titleTextStyle: {italic: false, bold: true}},
colors: ['cf513e']
});
print(chart);
We sometimes get error messages when creating a histogram, which might be because:
region
parameter, because GEE cannot calculate on an unbounded image.maxPixels
parameter. The defaulted maxPixels
is 1e6, which is too small for images with high spatial resolutions.Also note that the scale
parameter should match the spatial resolution of the image used to create a histogram. Otherwise, resampling algorithms will be automatically performed, which is not what we often want.
User memory limit is one of the greatest challenges. GEE does not allow us to perform infinite calculations in a single script, because that may occupy too many resources. As a result, we will get “User Memory out of Bound” error when generating charts or maps. In an extreme case, when we want to export our images to Asset or Google Drive, the running Task never ends. It should be noted that if a GEE Task runs for more than 24h, then it might never ends. Normally, a GEE Task should last for around or less than 1h. If not, you might have to manage the user memory limits issues.
The basic idea is to divide steps into different GEE scripts. For example, instead of running four steps in a single GEE script, we may run the first and the second steps in Script 1, export the image we get from Script 1, use the export as input in Script 2, and finally perform the third and the fourth steps in Script 2. The output of Script 2 is the final result we are looking for.
Therefore, the main problem is simply how to export an image or an image collection to our Asset. To export an single image (and this is obviously simpler than an image collection), we can use:
Export.image.toAsset({
image: image_name,
description: 'TASK_name',
assetId: 'projects/user-name/assets/file-name',
scale: 500,
region: NorthAmerica,
maxPixels: 1e13,
crs: 'EPSG:4326'
});
// `scale`: Spatial resolution of the exported image. 500 for 500m
// `region`: Study region
// `maxPixels`: Maximal number of pixels in the region
// `crs`: Coordinate system. It can be checked through `print(image_name)`
For image collections, we cannot export them directly. However, we can
For image collections, we cannot export them directly. However, we can export every single image in the collection using the following code:
// Export all of the images in an image collection
// Reference: https://github.com/fitoprincipe/geetools-code-editor
var batch = require('users/fitoprincipe/geetools:batch');
batch.Download.ImageCollection.toAsset(image_collection_name, 'asset_directory_name',
{scale: 500,
region: NorthAmerica,
maxPixels: 1e13});
// `scale`: Spatial resolution of the exported image. 500 for 500m
// `region`: Study region
// `maxPixels`: Maximal number of pixels in the region
When we need to import the image collection in Script 2, simply use the function ee.ImageCollection()
to pack all of the images into a collection.