Page 2 of 3

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Wed Feb 11, 2015 14:00
by Evergreen
s-l-teichmann wrote:Good news. Just got the confirmation from the committee: I will give a talk

"MTSatellite - Echtzeit Webmapping für Minetest-Welten oder Spiel-Spass mit GIS" [1]

at the FOSSGIS 2015 [2] conference. Its the leading Free Software
and Open Source Geo Information Systems conference here in Germany.

The talk will be in German and I'm going to finish a small paper about it.
I think there will be a video recording for later online view, too. The focus
of the talk will be the persistent layer and the strategies to make the access
fast enough to serve as a backend suitable for online live mapping.

... And of cause I will promote Minetest! :-)

I will give some more details as soon as I know more.

[1] "MTSatellite - realtime we mapping of Minetest worlds or gaming fun with GIS"
[2] http://www.fossgis.de/konferenz/2015/

Awesome!

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Thu Feb 19, 2015 15:34
by s-l-teichmann
To keep you updated: I've submit the paper for the conference proceedings last week. The talk will be 2015-03-12 at 14:30 in Münster [1].

I will try to cut out an 'official' release of MTSatellite in the next days as is has proven to be stable on our production MT server.

Feedback from other users is always welcome. :-)

[1] http://www.fossgis.de/konferenz/2015/pr ... 52.de.html
(Page looks ugly by now but I hope this will improve a bit.)

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Fri May 22, 2015 07:59
by 4aiman
What should I do if I get "404 page not found" instead of the map?

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Fri May 22, 2015 08:17
by s-l-teichmann
Hi!

I guess you're talking to the mtwebmapper server?!
If this is the case have you set its command line argument

-web=/path/to/your/static/web

to a proper path with the checkout of

https://bitbucket.org/s_l_teichmann/mts ... apper/web/

?

Try an absolute path to be sure you are point it to the right directory.
Hope that helps.

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Mon May 25, 2015 09:01
by 4aiman
I don't know how I've missed that setting.
Also I've figured it out that the map's tiles haven't been generated properly.

But now it works just fine.

Thank you, s-l-teichmann!
Regards!

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Tue May 26, 2015 08:09
by s-l-teichmann
Uploaded a YouTube video demo-ing the use of MTSatellite as a mini map:
https://www.youtube.com/watch?v=8e6eCMXguu8

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Tue May 26, 2015 10:24
by rubenwardy
That's running in a different window, right?

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Tue May 26, 2015 11:57
by ExeterDad
Doesn't seem to be a separate window! It's always on top. And Minetest wouldn't release the mouse pointer to access it. It's seamless!
Holy Crap that's awesome!

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Tue May 26, 2015 12:34
by rubenwardy
Could have been on a different computer.

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Wed May 27, 2015 10:23
by Minetestforfun
Player markers is a new feature ?! How do that ? :D
Can we add markers like "static marker" for buildings/areas of interrest now ? How do that ? :D

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Sun May 31, 2015 11:38
by s-l-teichmann
@Rubenwardy:

It's a separate window with window decoration turned off and set transparent by compositing.

@Minetestforfun:

Yes, MTSatellite is able to show players now.

You basically need a fresh versions of mtwebmapper, the static web stuff and the track_players-mod
coming with MTSatelliet. See [1] how to set it up.

We use it for about two month on our server without any problems but I still call it experimental
(and a bit rough around the edges). Also nice with in combination with the websocket socket
support so you don't have to refresh the map to update it manually.

Feel free to experiment with it. Feedback is welcome. :-)

Adding other stuff like areas, points of interest and other static content is possible.
You just have to produce layers of GeoJSON objects and add this to the map in the web client.
To do so there is still the need to write some external scripts.

BTW: The new client has leaflet-hash support which make linking
map sections easy: http://maps.mt.sha-bang.de/#10/-233.2500/-2067.7500

[1] https://bitbucket.org/s_l_teichmann/mts ... in-players

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Thu Jun 04, 2015 13:15
by s-l-teichmann
MTSatellite for Windows

(Very, very, very experimental and maybe very buggy)

I've uploaded a Windows build of MTSatellite to [1].
This is highly experimental and not tested very well. I've only run it
in a VirtualBoxed Windows 7 as I don't have any native Windows
machines around.

If someone likes to test it there is a little README.txt in the zip archive
how to set it up. To run it you need a Windows build of Minetest server
with Redis support compiled in, too.

Feedback welcome! :-)

Have I've said that it's very, very, very experimental and may contain bugsl?!

[1] https://bitbucket.org/s_l_teichmann/mts ... -06-04.zip
SHA1: 45977758121c54ebb604852180a4ea66f40fe752

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Thu Jun 04, 2015 14:48
by addi
in the readme you have written:
. Open a command line prompt:

SET PATH=%PATH%;C:\TEMP\mtsatellite-win\bin-win64

you dont need this step, if you navigate to the folder, and right-click while holding shift. Then select open command promt here.

1. Convert your world data to the interleaved LevelDB format:

Assume your game data resides at C:\GAMES\MT\worlds and
you have a World called 'demo' with a plain SQLite3 backend
configured:

mtdbconvert C:\GAMES\MT\worlds\demo\map.sqlite C:\GAMES\MT\worlds\demo\map.db

Depending on your world size this will take some time.

the correct command is mtdbconverter C:\GAMES\MT\worlds\demo\map.sqlite C:\GAMES\MT\worlds\demo\map.db

this worked great :-)
2. Start mtredisalize in another shell (dont forget to set the path):

SET GOMAXPROCS=4
mtredisalize -interleaved=true -network=tcp4 -change-url=http://localhost:8808/update \
C:\GAMES\MT\worlds\demo\map.db

I had some trouble with that backslash-linebreak .
So, i used:
SET GOMAXPROCS=4
mtredisalize -interleaved=true -network=tcp4 -change-url=http://localhost:8808/update C:\GAMES\MT\worlds\demo\map.db
this also worked great :-)

Maybe Im just too stupid, the mtmapseeder seems not to work. The files does nowhere appear and its running since 15 minutes for a small test world
Your phone or window isn't wide enough to display the code box. If it's a phone, try rotating it to landscape mode.
Code: Select all
2015/06/04 16:25:41 Writing (-1438, -94) to file map\8\31\115.png

Edit: I found the images now, but they are just white. they are saved to D:\Programme\mtsatellite-win-2015-06-04\bin-win64\map
Edit2:the file D:\Programme\mtsatellite-win-2015-06-04\bin-win64\map\0\0\0.png contains some pixels, that may be a map.

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Thu Jun 04, 2015 15:51
by s-l-teichmann
@addi

Thank you for your feedbak about the mistakes in the README.txt.
I'll correct them and upload a new version. :-)

About the white images and the directory structure generated by mtseeder:

\8\* Is the base set of images. In this folder and it's subfolders where every pixel represents
a node of the map.
\7\* Each pixels represents two nodes
\6\* Each pixels represenrs four nodes
... and so on
\0\0\0.png is the whole map averaged into a 256x256 image.

This is the tile pyramid which the leaflet client + mtwebmapper use
to do the zooming stuff. So your 0.png looks right to me.

If you have a small world with only a little land discovered around the
center most of the images are blank.
\8\120\120.png (middle of level 8) is likely to be not empty.

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Sat Jun 27, 2015 09:13
by rjregina
I'm going to write some documentation to setup and wire the system ... just a matter of spare time.
An idea (for the admins) : add a real-time moving pixel for each player connected and display a
> little bubble above that shows his coords and his last actions when you click on a player.
Sorry for only supporting GNU/Linux at the moment but this is where our 'productive' Minetest server runs o

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Thu Jul 16, 2015 08:21
by Minetestforfun
Hi,

Thank you again for your work, but I have some minor problems with the new mapper version...

Feel free to help us and gain our gratitude :D https://github.com/Ombridride/minetest- ... /issues/71

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Thu Jul 16, 2015 19:05
by s-l-teichmann
Have you solved the problem? When looking at http://minetestforfun-worldmap.qbuissondebon.info I see players.

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Sat Jul 18, 2015 18:44
by Minetestforfun
Yes, i solved this problem, it was due to our old "web" folder...

Now we have others problems/improves we need, if you want to help us => https://github.com/Ombridride/minetest- ... /issues/71

Thank you in advance !

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Sun Jul 19, 2015 13:01
by Hybrid Dog
can you make it work in 3d https://github.com/xyzz/onomatopoeia

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Sun Jul 19, 2015 23:07
by s-l-teichmann
@HybridDog Me and my guys already talked about isometric rendering and how to integrate it in MTSatellite. Last time I've checked the onomatopoeia script was awfully slow. Compared to the Minecraft counterparts isometric rendering in Minetest can be tricky. For example you can build a high tower in the corner next to the bottom of the map (which it is depends on the orientation of the map). This tower can shadow nodes deep in the map. Finding the map tiles to be updated is not trivial and needs a clever way of indexing. We don't have a perfect solution yet but the discussion continues.

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Wed Jul 22, 2015 08:43
by Minetestforfun
@s-l-teichmann

If it interrest you (colors.txt generator mod/python script) : https://github.com/Ombridride/minetest- ... ppercolors
made by Gravgun.

This solution check every nodes and auto generate its color (no unknow nodes with this script)

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Mon Aug 03, 2015 20:08
by s-l-teichmann
@Minetestforfun Thanks for the hint and thumbs up to Gravgun for writing the scripts.
Great starting point! :-)

I had a look at the mod+post processing tool at the weekend.
The post processor produces too much transparent entries (eg. grass). It also produces a transparent column next to the alpha channel in colors.txt which is not parsed by MTSatellite.
After some experimentation I ended in modifying the mod slightly to write out the draw types
of the nodes out too. This helps to determine if a node should lead to transparent entry or not.
Furthermore I wrote my own post processing tool which is about 6-10 times faster [1]
than the original Python script.

[1] https://bitbucket.org/s_l_teichmann/mtautocolors

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Tue Aug 25, 2015 19:40
by gravgun
@s-l-teichmann You just heavily butchered my nickname on your project description...

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Tue Aug 25, 2015 22:09
by s-l-teichmann
@gravgun Oh sorry, that was only an unintended typo. I've just corrected it.

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Fri Mar 11, 2016 18:43
by lightonflux
What do i need to do to limit everything to -4500 - 4500 as this is the worldedge limit of our server. This includes mtseeder.

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Fri May 06, 2016 15:23
by Minetestforfun
Hi,

Any news about MTSatellite ?

Im mainly looking for performance improvments about the mapper :)
(actually 50% of our servers CPU are used for mapper and 20% for the minetest-server processes, we manage 4 servers...)

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Sun May 08, 2016 15:00
by s-l-teichmann
@Minetestforfun

Im mainly looking for performance improvments about the mapper :)


Yes. There is an experimental branch [1] which SHA1 checksums
regenerated base tiles and does not write them to disk if they don't change.
This safes a lot of disk I/O and CPU too, because the pyramid tiles depending
on these unchanged base tiles don't get updated/scaled down either.
It also results in lesser notification on the websockets.

The code works but it needs a little clean up. I'll merge it back into master
soon. It would be nice to know if it improves your situation.

[1] https://bitbucket.org/s_l_teichmann/mts ... -overwrite

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Tue May 10, 2016 09:22
by Minetestforfun
Happy to see your answer, I will wait the code cleanup before use it, to be safe for all of our servers :)

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Fri May 13, 2016 14:52
by s-l-teichmann
I've merged the feature back. The tests on my local instance look pretty good.

We will upgrade our production server to the new version of MTSatellite in a couple of days (when doing the 0.4.14 update of the MT server).

BTW: MTSatellite have learned to set the background color [1]. This may be interessing for your Skyblock server?!

[1] https://bitbucket.org/s_l_teichmann/mts ... se-default

Re: MTSatellite: A "realtime" web mapping system for Minetes

PostPosted: Sun May 15, 2016 21:30
by Minetestforfun
Cool for the production tests, i will still wait a couple of days before updating MTSatellite on all of our servers :) Thank you for the return !

For the background color, it's a good new feature, but I'm hesitating if i change the Skyblock background color or not. Indeed the background is "white" as the "clouds" and the islands are above clouds so... I need some tests with different colors ;)