WorldPainter support?

User avatar
Captain_Chaos
Member
 
Posts: 42
Joined: Sun Feb 12, 2017 13:04
GitHub: Captain-Chaos

WorldPainter support?

by Captain_Chaos » Sun Feb 12, 2017 13:13

Hello everyone,

I'm the author of WorldPainter, an interactive map generator for Minecraft. I've been asked a few times whether I could support Minetest as well, and decided I'd look into it.

One thing I want to do is gauge the level of interest. Are many people interested in this? Also, are there already other tools for Minetest which do the same or similar things?

Another question is whether someone can direct me to information about the file format. I can probably figure it out from the source code, but if there is any documentation out there describing the details that might speeds things up considerably.

Thanks in advance for any answers and information you can give!

Cheers,
Captain_Chaos
 

User avatar
Krock
Member
 
Posts: 3598
Joined: Thu Oct 03, 2013 07:48
GitHub: SmallJoker

Re: WorldPainter support?

by Krock » Sun Feb 12, 2017 16:13

Hello and welcome to Minetest.

Do I see it right that this mod is able to use images to generate maps, similar to this mod?
The videos I've seen look very promising, with the advanced terrain editor that allows users to easily draw and generate maps. I'm interested in map generation and especially in generating custom maps that do not base on perin noise - to get a variety. Hopefully other people think so too.

What do you mean with "direct me to information about the file format"? If you're looking for the method to generate terrain, you will find the functions in the lua_api.txt file and further descriptions in the wiki.
Newest Win32 builds - Find a mod - All my mods
ALL YOUR DONATION ARE BELONG TO PARAMAT (Please support him and Minetest)
New DuckDuckGo !bang: !mtmod <keyword here>
 

User avatar
Captain_Chaos
Member
 
Posts: 42
Joined: Sun Feb 12, 2017 13:04
GitHub: Captain-Chaos

Re: WorldPainter support?

by Captain_Chaos » Sun Feb 12, 2017 17:25

Krock wrote:Do I see it right that this mod is able to use images to generate maps, similar to this mod?

I'm not sure what that mod does, but I'll explain more fully what WorldPainter does: WorldPainter is not a mod, but a stand alone program, with which you generate maps for Minecraft. So instead of Minecraft generating the blocks, or Minecraft mods generating the blocks, WorldPainter generates them according to your instructions about the terrain height and type, and any additional "layers" you've painted which can add things like trees and other vegetation, caves or any type of custom object.

You can paint and sculpt the terrain by hand, but you can also use height maps and/or image masks, which I think answers your question.

Krock wrote:What do you mean with "direct me to information about the file format"?

WorldPainter is a stand alone program, which pre-generates all the blocks for the entire map and stores them on disk in Minecraft's native format. In order to support Minetest I would have to know what Minetest's on-disk file format is so I can do the same for it. WorldPainter is not a mod, and as such does not run inside Minetest and cannot make use of its runtime APIs. In theory something like that might be possible, but an additional hurdle for that is that Minetest is written in C++ and WorldPainter in Java. So the best alternative is probably for WorldPainter to directly write the blocks to disk in Minetest's native block storage format. What I'm looking into now is what that format is and therefore how feasible that would be.

I think I have seen that Minetest actually supports different backends for storing the blocks. Is there a de facto standard which WorldPainter could support and be useful to most users? Can one of the formats easily be converted to the other formats so that it would make sense to support that format? Is there any detailed documentation about the technical implementation details (data models, data structures, etc.) of these storage formats/backends?
 

Nore
Member
 
Posts: 468
Joined: Wed Nov 28, 2012 11:35
GitHub: Ekdohibs

Re: WorldPainter support?

by Nore » Sun Feb 12, 2017 17:40

The format is more or less always the same, (almost) the only thing that changes is how the blocks are stored in the db. More precisely, each 16*16*16 block is first serialized, then inserted into the db, the serialization being the same for all db backends. The most used one is sqlite3, as it was originally the only one Minetest supported, so you can be sure most if not all users will be able to open such maps.

As for the precise block format, there unfortunately is no documentation that I know of. However you can look at some code producing it, such as https://github.com/minetest/minetest/blob/master/src/mapblock.cpp#L556 or https://github.com/minetest-tools/mcimport/blob/master/block.py#L344.

Also, beware the following differences between MC and MT:
- The x and z axes do not work exactly as they work in MT: if you proceed the same way, then the map will be flipped.
- Light is stored as light on day and night, not as sunlight and artificial light.
 

User avatar
Krock
Member
 
Posts: 3598
Joined: Thu Oct 03, 2013 07:48
GitHub: SmallJoker

Re: WorldPainter support?

by Krock » Sun Feb 12, 2017 17:46

PRE-EDIT: Nice ninja, Nore ;)

Thanks for your explanation. I assumed it would be imported with a mod - but actually you're writing the stuff directly to the database.
There are multiple backends, such as SQLite3, LevelDB, Redis and PostgreSQL (sorted by addition date, where SQLite3 is the longest and likely the most used one). A full documentation of how the database structure looks like can be found in the file doc/world_format.txt. This structure is the same for all backends, only the functions to insert, delete or update the data varies.

C++ code that could be helpful:
https://github.com/minetest/minetestmap ... r.cpp#L362
This tool reads the database contents and generates a map from it. My knowledge about this is near to zero, thus it might be better to ask some core developers on freenode/#minetest-dev.
Newest Win32 builds - Find a mod - All my mods
ALL YOUR DONATION ARE BELONG TO PARAMAT (Please support him and Minetest)
New DuckDuckGo !bang: !mtmod <keyword here>
 

User avatar
octacian
Member
 
Posts: 408
Joined: Mon Dec 21, 2015 22:18
GitHub: octacian
IRC: octacian
In-game: octacian

Re: WorldPainter support?

by octacian » Sun Feb 12, 2017 17:48

Pre-edit: oh, looks like his response is already complete :rotfl:

Just FYI, I for one would be very interested in seeing something like this. However, the MT developers can give a lot more info on this than I can, so I'll leave it to them (and Krock, seeing as told me he was preparing a response).
God isn't dead!

My Coolest Mods:
MicroExpansion, Working Computers, Interchangeable Hands

Check out my YouTube channel! (octacian)
 

User avatar
Captain_Chaos
Member
 
Posts: 42
Joined: Sun Feb 12, 2017 13:04
GitHub: Captain-Chaos

Re: WorldPainter support?

by Captain_Chaos » Sun Feb 12, 2017 17:55

Thanks guys, this is all very useful and exactly what I was looking for. I'll look into how feasible it would be to support Minetest.

How do the available blocks compare between Minetest and Minecraft? Do the blocks have numerical ids similar to those of Minecraft? Is there a handy overview somewhere of all the blocks Minetest knows about? Are there similar mechanisms for more complex blocks containing data, such as chests, and for mobs?
 

Nore
Member
 
Posts: 468
Joined: Wed Nov 28, 2012 11:35
GitHub: Ekdohibs

Re: WorldPainter support?

by Nore » Sun Feb 12, 2017 18:01

There are no blocks id per se, all mapblocks contain something called "name-id mapping", that explain the mapping between the id of a node in the mapblock, and its name (like, "default:stone" for stone). To know the name of the various nodes, you can look that up on the wiki (http://wiki.minetest.net/Stone). Finally, for chests and the like, their additional information is stored in metadata; the easiest way to understand the format for MC-like nodes like chests would perhaps be to look at the code of mcimport. For mobs, well... there are no mobs in the "default" game (Minetest Game), so you need to look at some mob mod for that.
 

User avatar
Captain_Chaos
Member
 
Posts: 42
Joined: Sun Feb 12, 2017 13:04
GitHub: Captain-Chaos

Re: WorldPainter support?

by Captain_Chaos » Sun Feb 12, 2017 18:08

Ah, interesting. That is something that is sorely missing in Minecraft's file format, but it does mean some significant changes for WorldPainter, as it is based on the assumption of a static block to id mapping. Nevertheless it seems doable, especially if I leave mod support for later.
 

twoelk
Member
 
Posts: 1092
Joined: Fri Apr 19, 2013 16:19

Re: WorldPainter support?

by twoelk » Mon Feb 13, 2017 04:30

Wow this is great news.
It would be wonderfull if the WorldPainter could export to Minetest. I used the program with Minecraft and have used it to create content for Minetest by using some of the converters listed below

mcimport - import Minecraft classic maps by Sokomine
Converter Minecraft world to Minetest world by dgm5555
Convert parts of Minecraft maps to Minetest WorldEdit schematics by sfan5
Convert Minecraft maps to Minetest worlds by sofar
Minecraft to Minetest Block Converter (mcblocks) by dgm5555
compatibility nodes for MC schematics [mccompat] by Sokomine
fork of mcimport by dgm3333

I guess for a start this would be the easiest way as the node organization between mt and mc differs quite a lot. Basically mc has a fixed set of posible nodes if I remember correctly, while mt has a very open organization. Anybody can and countless have added mods for minetest which then have added an completetly unorganized number of new nodes to mt that number in the thousands. There is no central list and some are unique to single servers which may or may not have published the relevant source code.

So as stated above the easiest way would probably be for a start to be able to export a map created as for mc to the mt format. I would also suggest to stick to sqlite for a start as the backend formats can be converted to other formats by mt itself quite easily.

Minetest is organized as an engine that is expanded by some Lua scripts. The game content itself such as nodes and mobs is provided as mods. A subgame is a collection of mods that are designed to work well together and provide a complete gameplay in contrast to just a collection of mods such as mod-packs that might provide a group of mods about a certain subject or feature. So in contrast to mc there is no standalone version of mt. Minetest is always a combination of "Minetest the engine" that runs a "Minetest subgame" and this may or may not also include some more mod-packs or single mods. The Minetest Server List might give an insight to the great mods variety. You may view some of the mods listed that are installed by hovering along the column named "Version, Subgame, Mapgenerator"

There have been some efforts to create subgames that either recreate the gameplay of mc or to allow easily to play maps converted from mc. MineClone 2 by Wuzzy and others is the most recent project that might be of interest here. Others may be found in this List of Subgames

MinetestMapper is a group of programs that can draw overview maps of minetest worlds, and as such can be used to analyze them. For example if you run the MinetestMapperGui - Graphical userinterface for mtmapper you can create a colors.txt file that could be simply used as node list. If you supply an empty colors.txt file and try to map a world the program will complain and provide a complete list of all nodes present in the map. A very large but far from complete colors.txt file that focuses on the "Dreambuilder Modpack" that contains many of the more popular mods can be found on digitalaudioconcepts or another that focuses on the Ethereal mods is the xaagle-colorfile.
 

User avatar
Captain_Chaos
Member
 
Posts: 42
Joined: Sun Feb 12, 2017 13:04
GitHub: Captain-Chaos

Re: WorldPainter support?

by Captain_Chaos » Mon Feb 13, 2017 14:40

Thanks for the extensive info, twoelk!

It sounds like I would have to support Minetest mods in WorldPainter for it to have any practical use. Would you agree? Although I do understand there is a default subgame with the basic Minecraft-like blocks in it (the blocks I see if I just download and run Minetest); would it be of any use to support only those blocks, at least initially, or is the actual use of that block set very limited in practice?
 

User avatar
TheReaperKing
Member
 
Posts: 493
Joined: Sun Nov 22, 2015 21:36

Re: WorldPainter support?

by TheReaperKing » Mon Feb 13, 2017 16:48

I'm not an expert in this area but from what I've seen most mods actually depend on the minetest_game blocks and resources. In my opinion you could definitely initially start off with supporting those blocks and they are most common such as dirt and trees and such.

Also this sounds super exciting and useful for my students as well as we could recreate existing and past areas very quickly or even our own new creations. Thank you for taking an interest in supporting Minetest!
Project Lead of the Doom 3 Mod Last Man Standing - http://Doom3Coop.com

Project Lead of Platinum Arts Sandbox Free 3D Game Maker - http://SandboxGameMaker.com

Youtube Channel - https://www.youtube.com/user/PlatinumArtsKids
 

User avatar
toby109tt
Member
 
Posts: 296
Joined: Sun May 29, 2016 23:42
GitHub: tobyplowy
In-game: Toby109tt

Re: WorldPainter support?

by toby109tt » Mon Feb 13, 2017 18:55

Thanks Captain_Chaos for your interest in minetest It really makes me happy!
( ͡° ͜ʖ ͡°) i love pixels and voxels ( ͡° ͜ʖ ͡°)
 

sofar
Member
 
Posts: 781
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: WorldPainter support?

by sofar » Mon Feb 13, 2017 19:48

Captain_Chaos wrote:Thanks for the extensive info, twoelk!

It sounds like I would have to support Minetest mods in WorldPainter for it to have any practical use. Would you agree? Although I do understand there is a default subgame with the basic Minecraft-like blocks in it (the blocks I see if I just download and run Minetest); would it be of any use to support only those blocks, at least initially, or is the actual use of that block set very limited in practice?


The `factory` game which most mods rely on is `minetest_game`. (github.com/minetest/minetest_game)

Extracting the blocks is relatively simple because we can easily run some quick code in minetest and have it spit out a list:

Put this in ~/.minetest/worlds/test/worldmods/test/init.lua:

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.
Code: Select all
minetest.after(0, function()
   for k, _ in pairs(minetest.registered_nodes) do
      print(k)
   end
end)


And it spits out the full list that your minetest currently understands.

My output for 0.4.15 minetest:

https://gist.github.com/sofar/1c9475bea ... 25f89ed669
 

sofar
Member
 
Posts: 781
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: WorldPainter support?

by sofar » Mon Feb 13, 2017 19:52

I think you could probably settle on a basic set of landscape blocksnodes (dirt/stones, plants, liquids, etc.) to start with since that will already get you all the blocks that most people need. The rest of the blocks could be optionally input by a user.

We could also make some trivial tooling to output these lists (and add e.g. colors to the exported list) so that adding more would become trivial for users to do themselves.
 

User avatar
D00Med
Member
 
Posts: 712
Joined: Sat Feb 07, 2015 22:49
GitHub: D00Med

Re: WorldPainter support?

by D00Med » Mon Feb 13, 2017 20:54

This would be a really great thing to have! Thanks for considering supporting minetest
Look! I have a signature :]
My subgame: https://forum.minetest.net/viewtopic.php?f=15&t=14051#p207242
dmobs2 is coming...
 

User avatar
TumeniNodes
Member
 
Posts: 1335
Joined: Fri Feb 26, 2016 19:49
GitHub: TumeniNodes

Re: WorldPainter support?

by TumeniNodes » Tue Feb 14, 2017 02:31

This is a very nice gesture.
And I think the interest is fairly solid. And if the option does become available the interest will only grow from there.
Flick?... Flick who?
 

User avatar
sfan5
Member
 
Posts: 3636
Joined: Wed Aug 24, 2011 09:44
GitHub: sfan5
IRC: sfan5

Re: WorldPainter support?

by sfan5 » Tue Feb 14, 2017 12:10

Mods: Mesecons | WorldEdit | Nuke
Minetest builds for Windows (32-bit & 64-bit)
 

User avatar
Captain_Chaos
Member
 
Posts: 42
Joined: Sun Feb 12, 2017 13:04
GitHub: Captain-Chaos

Re: WorldPainter support?

by Captain_Chaos » Tue Feb 14, 2017 17:56

Another question: I see that there are is a whole list of different versions of the serialization format. Could I get away with supporting only one version, for instance the latest one? Is there a current de facto standard version which most people are using? Or are people using wildly different versions?
 

Nore
Member
 
Posts: 468
Joined: Wed Nov 28, 2012 11:35
GitHub: Ekdohibs

Re: WorldPainter support?

by Nore » Tue Feb 14, 2017 18:04

I suggest you use the oldest version that supports all the features you want; version 25 may be a good choice since it is already quite old (at least a few years), while version 27 was introduced just a few days ago to fix lighting problems, so pre-v27 mapblocks may be slightly slower to load on new builds (but only the first time). If you want to support old versions, such as the stable one, I think it is best to write v25 maps; if you care less about that, then you can support the lastest version, but only new servers will be able to open the maps.
(Also, there is a version 26, but I don't remember the difference with version 25.)
 

User avatar
toby109tt
Member
 
Posts: 296
Joined: Sun May 29, 2016 23:42
GitHub: tobyplowy
In-game: Toby109tt

Re: WorldPainter support?

by toby109tt » Tue Feb 14, 2017 21:06

Nore wrote:I suggest you use the oldest version that supports all the features you want; version 25 may be a good choice since it is already quite old (at least a few years), while version 27 was introduced just a few days ago to fix lighting problems, so pre-v27 mapblocks may be slightly slower to load on new builds (but only the first time). If you want to support old versions, such as the stable one, I think it is best to write v25 maps; if you care less about that, then you can support the lastest version, but only new servers will be able to open the maps.
(Also, there is a version 26, but I don't remember the difference with version 25.)


Shouldn't he support v0.4.15 aka the newest stable
( ͡° ͜ʖ ͡°) i love pixels and voxels ( ͡° ͜ʖ ͡°)
 

User avatar
Captain_Chaos
Member
 
Posts: 42
Joined: Sun Feb 12, 2017 13:04
GitHub: Captain-Chaos

Re: WorldPainter support?

by Captain_Chaos » Tue Feb 14, 2017 21:41

And one more. :) Minetest has negative z (vertical) coordinates if I understand correctly, which Minecraft doesn't, so that would also be a challenge. Is the water level at z == 0? When does Minetest generate new map blocks? Is it in a sphere around the player? What is the default radius in which it will generate map blocks?
 

Nore
Member
 
Posts: 468
Joined: Wed Nov 28, 2012 11:35
GitHub: Ekdohibs

Re: WorldPainter support?

by Nore » Tue Feb 14, 2017 21:45

Default water level is z = 0. About generating new mapblocks, it happens when the player is close enough; I think it can be configured but I don't remember the default radius. Also, the map is generated in mapchunks of 5*5*5 mapblocks (that's the default; if you produce your own maps, it is advised to set this number (chunksize or something like this IIRC) to 1, and to set mapgen to "singlenode" so that the map generation will only generate air and will node interfere with your map generation).
 

paramat
Member
 
Posts: 2662
Joined: Sun Oct 28, 2012 00:05
GitHub: paramat

Re: WorldPainter support?

by paramat » Wed Feb 15, 2017 03:09

In MT X is Eastwards, Z is Northwards and Y is up.
(0,0,0) is world centre and the world extends 32000 nodes (we call the basic unit-cubes in the world 'nodes') in all 6 directions.
Sea level is at Y = 1.
Our 'mapblocks' are cubic, 16 * 16 * 16 nodes in size.
World is generated in a cube around the player default radius 6 * 16 = 96 nodes, but of course world is generated in batches of mapblocks called 'mapchunks', each 5 * 5 * 5 mapblocks in size, so i assume whenever the 96 node cube around the player touches a mapchunk that mapchunk is generated.
 

User avatar
Captain_Chaos
Member
 
Posts: 42
Joined: Sun Feb 12, 2017 13:04
GitHub: Captain-Chaos

Re: WorldPainter support?

by Captain_Chaos » Wed Feb 15, 2017 16:37

Questions, questions... The world_format.txt file describes a list of files which make up a map, in addition to the block database. Files such as auth.txt, env_meta.txt, ipban.txt, etc.

What would be the absolute minimum I could get away with for generating these files? In other words, which could I leave out because they are optional or Minetest will generate them itself if they are missing, and of the files which must be present, what content could I leave out of them which will be supplemented by Minetest?
 

User avatar
Captain_Chaos
Member
 
Posts: 42
Joined: Sun Feb 12, 2017 13:04
GitHub: Captain-Chaos

Re: WorldPainter support?

by Captain_Chaos » Wed Feb 15, 2017 16:40

Also, I have the basic game running, but I'm trying to compile it myself because that would make it much easier to test my understanding of the source code. I'm having trouble though. I'm compiling it with MinGW, and it gets to the minetest object file stage, but it won't link with the precompiled binaries I downloaded of the libraries, presumably because they are meant for MSVC.

Are there precompiled versions of the libraries for MinGW? What is the easiest way to build Minetest (and minetest_game) on Windows 10, for someone not in possession of Microsoft Visual Studio?
 

User avatar
Captain_Chaos
Member
 
Posts: 42
Joined: Sun Feb 12, 2017 13:04
GitHub: Captain-Chaos

Re: WorldPainter support?

by Captain_Chaos » Wed Feb 15, 2017 17:38

What does Minetest do if it encounters unknown node names in the map? Will it crash, ignore them, or display them as some standard block? Will it keep the unknown node, or will it remove it?
 

User avatar
Krock
Member
 
Posts: 3598
Joined: Thu Oct 03, 2013 07:48
GitHub: SmallJoker

Re: WorldPainter support?

by Krock » Wed Feb 15, 2017 17:43

The libraries that are labeled as "pre-compiled libraries" are indeed compiled with MSVC (2010) for 32-bit MSVC builds only. I'd be surprised when one gets it working with MinGW.

Check out the buildbot script. That script and the 32-bit pendant download cross-compiled libraries for MinGW and compile the binary automatically.

Captain_Chaos wrote:What does Minetest do if it encounters unknown node names in the map? Will it crash, ignore them, or display them as some standard block? Will it keep the unknown node, or will it remove it?

In-game they get the "unknown node" texture. When you dig them, they will convert to an "unknown item", which can not be used anywhere but it's still possible to get its original name (whatever it was before it turned into an unknown thing).
Newest Win32 builds - Find a mod - All my mods
ALL YOUR DONATION ARE BELONG TO PARAMAT (Please support him and Minetest)
New DuckDuckGo !bang: !mtmod <keyword here>
 

twoelk
Member
 
Posts: 1092
Joined: Fri Apr 19, 2013 16:19

Re: WorldPainter support?

by twoelk » Wed Feb 15, 2017 17:51

+ some unknown blocks on a server
Attachments
screenshot_105553239.jpg
screenshot_105553239.jpg (375.76 KiB) Viewed 8830 times
 

sofar
Member
 
Posts: 781
Joined: Fri Jan 16, 2015 07:31
GitHub: sofar
IRC: sofar
In-game: sofar

Re: WorldPainter support?

by sofar » Wed Feb 15, 2017 19:08

Captain_Chaos wrote:In other words, which could I leave out because they are optional or Minetest will generate them itself if they are missing?


Everything is optional. I have a test case that runs a minetest server only from an empty folder. So all you'd have to do is create (on Linux) ~/.minetest/worlds/new_world/ and put stuff in there.

Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.
Code: Select all
$ mkdir ~/.minetest/worlds/new_world_test
$ minetestserver --world ~/.minetest/worlds/new_world_test --gameid minetest_game

(runs)
 

Next

Return to Minetest-Related

Who is online

Users browsing this forum: No registered users and 4 guests

cron