Prepared by: Tom Hengl (OpenGeoHub), Koen van Seijen (RegenEarth), Naeem Lakhani (The First Thirty), and Ichsani Wheeler (OpenGeoHub) Soil Health Now! was held at the Wageningen International…
Copernicus Land Monitoring Services provides Surface Soil Moisture 2014-present (raster 1 km), Europe, daily – version 1. Each day covers only 5 to 10% of European land mask and shows lines of scenes (obvious artifacts). This is the long-term aggregates of daily images of soil moisture (0–100%) based on two types of aggregation: Long-term quarterly (winter, spring, summer and autumn), Annual quantiles P.05, P.50 and P.95, The soil moisture rasters are based on Sentinel 1 and described in detail in: Bauer-Marschallinger, B. ; Freeman, V. ; Cao, S. ; Paulik, C. ; Schaufler, S. ; Stachl, T. ; Modanesi, S. ; Massari, C. ; Ciabatta, L. ; Brocca, L. ; Wagner, W. Toward Global Soil Moisture Monitoring With Sentinel-1: Harnessing Assets and Overcoming Obstacles. IEEE Transactions on Geoscience and Remote Sensing 2019, 1 - 20. DOI 10.1109/TGRS.2018.2858004 Aggregation has been generated using the terra package in R in combination with the matrixStats::rowQuantiles function. library(terra) library(matrixStats) g1 = terra::vect("/mnt/inca/EU_landmask/tilling_filter/eu_ard2_final_status.gpkg") ## 1254 tiles tile = g1[534] nc.lst = list.files('/mnt/landmark/SM1km/ssm_1km_v1_daily_netcdf/', pattern = glob2rx("*.nc$"), full.names=TRUE) ## 3726 ## test it #r = terra::rast(nc.lst[100:210]) agg_tile = function(r, tile, pv=c(0.05,0.5,0.95), out.year="2015.annual"){ bb = paste(as.vector(ext(tile)), collapse = ".") out.tif = paste0("./eu_tmp/", out.year, "/sm1km_", pv, "_", out.year, "_", bb, ".tif") if(any(!file.exists(out.tif))){ r.t = terra::crop(r, ext(tile)) ## each tile is 100x100 pixels 365 days r.t = as.data.frame(r.t, xy=TRUE, na.rm=FALSE) sel.c = grep(glob2rx("ssm$"), colnames(r.t)) ## remove everything outside the range t1s = cbind(data.frame(matrixStats::rowQuantiles(as.matrix(r.t[,sel.c]), probs = pv, na.rm=TRUE)), data.frame(x=r.t$x, y=r.t$y)) #str(t1s) ## write to GeoTIFFs r.o = terra::rast(t1s[,c("x","y","X5.","X50.","X95.")], type="xyz", crs="+proj=longlat +datum=WGS84 +no_defs") for(k in 1:length(pv)){ terra::writeRaster(r.o[[k]], filename=out.tif[k], gdal=c("COMPRESS=DEFLATE"), datatype='INT2U', NAflag=32768, overwrite=FALSE) } rm(r.t); gc() tmpFiles(remove=TRUE) } } ## quarterly values: lA = data.frame(filename=nc.lst) library(lubridate) lA$Date = ymd(sapply(lA$filename, function(i){substr(strsplit(basename(i), "_")[[1]][4], 1, 8)})) #summary(is.na(lA$Date)) #hist(lA$Date, breaks=60) lA$quarter = quarter(lA$Date, fiscal_start = 11) summary(as.factor(lA$quarter)) for(qr in 1:4){ #qr=1 pth = paste0("A.q", qr) rs = terra::rast(lA$filename[lA$quarter==qr]) #agg_tile(rs, tile, out.year=pth) x = parallel::mclapply(sample(1:length(g1)), function(i){try( agg_tile(rs, tile=g1[i], out.year=pth) )}, mc.cores=20) for(type in c(0.05,0.5,0.95)){ x <- list.files(path=paste0("./eu_tmp/", pth), pattern=glob2rx(paste0("sm1km_", type, "_*.tif$")), full.names=TRUE) out.tmp <- paste0(pth, ".", type, ".sm1km_eu.txt") vrt.tmp <- paste0(pth, ".", type, ".sm1km_eu.vrt") cat(x, sep="\n", file=out.tmp) system(paste0('gdalbuildvrt -input_file_list ', out.tmp, ' ', vrt.tmp)) system(paste0('gdal_translate ', vrt.tmp, ' ./cogs/soil.moisture_s1.clms.qr.', qr, '.p', type, '_m_1km_20140101_20241231_eu_epsg4326_v20250206.tif -ot "Byte" -r "near" --config GDAL_CACHEMAX 9216 -co BIGTIFF=YES -co NUM_THREADS=80 -co COMPRESS=DEFLATE -of COG -projwin -32 72 45 27')) } } ## per year ---- for(year in 2015:2023){ l.lst = nc.lst[grep(year, basename(nc.lst))] r = terra::rast(l.lst) ## test it: pth = paste0(year, ".annual") x = parallel::mclapply(sample(1:length(g1)), function(i){try( agg_tile(r, tile=g1[i], out.year=pth) )}, mc.cores=40) ## Mosaics: for(type in c(0.05,0.5,0.95)){ x <- list.files(path=paste0("./eu_tmp/", pth), pattern=glob2rx(paste0("sm1km_", type, "_*.tif$")), full.names=TRUE) out.tmp <- paste0(pth, ".", type, ".sm1km_eu.txt") vrt.tmp <- paste0(pth, ".", type, ".sm1km_eu.vrt") cat(x, sep="\n", file=out.tmp) system(paste0('gdalbuildvrt -input_file_list ', out.tmp, ' ', vrt.tmp)) system(paste0('gdal_translate ', vrt.tmp, ' ./cogs/soil.moisture_s1.clms.annual.', type, '_m_1km_', year, '0101_', year, '1231_eu_epsg4326_v20250206.tif -ot "Byte" -r "near" --config GDAL_CACHEMAX 9216 -co BIGTIFF=YES -co NUM_THREADS=80 -co COMPRESS=DEFLATE -of COG -projwin -32 72 45 27')) } }
The paper describes a comprehensive framework for soil organic carbon density (SOCD) (kg/m3) modeling and mapping, based on spatiotemporal Random Forest (RF) and Quantile Regression Forests (QRF). 22,428 SOCD measurements and a wide range of covariate layers—particularly the 30m Landsat-based ...
I often wonder if what I do helps the world, enough?
I remember @harari_yuval & Matan Gavish describing agriculture as a crucial field for social good.
This group bridges agricultural information gaps, with NLP!
https://arxiv.org/abs/2407.04721
Agriculture, vital for global sustenance, necessitates innovative solutions due to a lack of organized domain experts, particularly in developing countries where many farmers are impoverished and cannot afford expert consulting. Initiatives like Farmers Helpline play a crucial role in such countries, yet challenges such as high operational costs persist. Automating query resolution can alleviate the burden on traditional call centers, providing farmers with immediate and contextually relevant information. The integration of Agriculture and Artificial Intelligence (AI) offers a transformative opportunity to empower farmers and bridge information gaps. Language models like transformers, the rising stars of AI, possess remarkable language understanding capabilities, making them ideal for addressing information gaps in agriculture. This work explores and demonstrates the transformative potential of Large Language Models (LLMs) in automating query resolution for agricultural farmers, leveraging their expertise in deciphering natural language and understanding context. Using a subset of a vast dataset of real-world farmer queries collected in India, our study focuses on approximately 4 million queries from the state of Tamil Nadu, spanning various sectors, seasonal crops, and query types.
Are you a data scientist, pedometrician and/or spatial modeler developing methods for monitoring #soil properties & supporting #landrestoration projects? We have 2 exciting hackathons for you:
1. "ESA EO4SoilProtection 2024: Predicting SOC Density"
https://www.kaggle.com/competitions/esa-eo4soilprotection-2024-predicting-soc-density/
2. "ESA EO4SoilProtection 2024: Predicting Erosion Categories"
https://www.kaggle.com/competitions/esa-eo4soilprotection-2024-predicting-erosion-cat/
The winners will be announced at the https://www.eo4soilprotection.org/ workshop at ESRIN. #OpenEarthMonitor #AI4SoilHealth
May the best team win!
Biodiversa+
OpenGeoHub Foundation
Tomislav Hengl
Leshem Choshen
