Gael de Sailly wrote:I have another idea:
making a separate python script that converts the image into raw data that Lua can easily use. Run the script before starting Minetest.
With this method, the raw data can even been split up into many different tiles, each of them matching a… maybe 200x200 pixels, hugely reducing the amount of used RAM (unload a tile when it has not been used during 5 consecutive chunk generations for example). We can even use zlib compression, performed by python, and decompressed by the Minetest API:
Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.
I mean, in brief, what has been done for the
Moon Mapgen mod by arpruss.
I have considered this approach but am not, personally, interested in it. The current method, with most of the (working) libraries does not load the entire image into memory anyway (the native png processor lifted from imageloader mod is an exception and crashes on even mid-sized images -- lua image libraries that load the entire image into memory were not considered for this mod. the imagemagick and graphicsmagic libs read the entire file and create a disk cache of it but at least do not load it into memory).
Mainly though I don't want to create any temp files because many of the source files of DEMs are huge and because I want them to be usable without any extra processing steps, otherwise we might as well convert DEM formats to ESRI's GRID (text-based) format. However, I envision a tiling system for version 0.2.0 of the mod which would allow any kind of tiles to be used, whether they be multiple contiguous tiles from a DEM set or from a python script as you say.
the mod already performs "caching" of sorts and only looks up pixels for map chunk "footprints" once in a vertical column -- it knows when a sky or sub-surface chunk is being rendered and does no pixel lookups...
I also experimented with cropping the image into smaller images for efficiency (not 200x200 but mapchunk size of 80x80 and multiples thereof) but with large source DEMs (which are the ones we'd want to do this with) re-opening the source file for cropping took several seconds each time. certain processors actually do this out of necessity. gm and convert for example -- which then translate the image into and parse a text pixel enumeration, which isn't working in windows...)
I think the best way forward is to use core's image handling functions to create lua methods that will open an image and get pixel values. anyone willing to help with this task please jump in ASAP. we could eliminate all outside libraries, command line (python) calls, and remove the mod security settings for this mod, making it much more portable and easy to use by a wider audience.