FOSS gamedev and creative worlds

Posted: **Fri Mar 07, 2014 00:05**

A utility that installs and updates mods from GitHub

Latest GitHub version as a ZIP file:
https://github.com/mdoege/MT-GitSync/archive/master.zip (v4.2; updated 2017-03-19)
GitHub repo:
https://github.com/mdoege/MT-GitSync
License: CC0

(For developers: Tools for maintaining the mod list are in the gitsync-devel repository.)

Screenshot

Features

Works on Windows, macOS (OS X), and Linux
1,627 mods in included database (subgames and texture packs are also added now, but they are not very complete yet)
You can also enter author/repository name manually
Dependencies get installed automatically
Also supports Gitorious/Bitbucket/etc. repository URLs
Click "Info" to open the forum topic for a mod in the web browser

You need:

Python (3.x or 2.7) from https://www.python.org/downloads/windows/ (Windows) or Homebrew ("brew install python", OS X)
git from http://git-scm.com/downloads or Homebrew ("brew install git"; OS X)
The git binary must be somewhere in your path. On Windows this is an option you have to select in the git installer!
On some Linux distributions such as Ubuntu, Tkinter is not installed by default, so python-tk (for 2.7) or python3-tk are needed too.

Basic usage

First set up the path to your MT mods directory by clicking on the "Set directory" button. Then you can either install a new mod (enter GH user name +"/" + repo name and click Install; clone URLs from others sites like Bitbucket or Gitorious also work) or update all mods. You can also select a mod from the built-in mod database.

"Info" opens the GitHub page for the currently selected mod. The "Sorted" checkbox toggles between the default view (sorted A-Z) and the most recent view (latest additions to the mod database at the top.) For some mods, ratings from Krock's MT mod database are displayed, e.g. "[+3/-1]" means three upvotes and one downvote. Repositories prepended with "0>>>" (at the top of the list by default) are already downloaded.

Instead of a mods directory, you could also point this utility e.g. to a "worldmods/" folder in a world directory. Or to your textures folder to clone and update texture packs from GitHub in this way. This tool is fairly general-purpose and works for other git repositories too, not just MT mods.

This program will show an error message in the activity log area if e.g. you try to add a mod that is already installed. So it should be pretty safe to use and not screw up you mods directory.

Notes for mod creators

Make sure git is configured not to pull if a different branch is checked out locally. This should be the standard safe behavior for new git installations, but older git configs might be set up for different default behavior.

A directory containing the file "._MTGITSYNC_IGNORE_" will be skipped during updates, so this is a way to protect mods you are developing yourself. After an update, the program will also print a summary about skipped mods. Don't forget to add this filename to your .gitignore however, otherwise the file will be uploaded to the GitHub repo too and will block updates for everyone else who uses MT-GitSync!

Posted: **Fri Mar 07, 2014 00:08**

Nice.

Have you made sure it is forward compatible?

Ie, using print (msg), which works on both versions.

Posted: **Fri Mar 07, 2014 00:12**

Yes. I only use a single print statement / function anyway. :-)

I actually started this as a Python 3 script, but tkFileDialog does not seem to exist in Python 3 anymore, at least I could not find it. Has it been renamed to something else? Very strange. But apart from Tk module name issues, the script should work in Python 3 unchanged I think.

P.S. Aha, according to 2to3 the correct import statement for Python 3 would have been "from tkinter.filedialog import *".

Posted: **Fri Mar 07, 2014 02:35**

I suggest using pygtk instead of tk. It looks nicer imo.

Posted: **Fri Mar 07, 2014 08:29**

Yes, but GTK would be another dependency, while Tkinter comes with Python. Linux has advanced package managers like pacman, so installing dependencies is not a problem, but on Windows and OS X they are quite annoying. And this thing is more geared towards Windows. Because on Linux I can just navigate to a mod folder with Dolphin, press F4, and type 'git pull'.

I may do an upgraded version, but I think I would be using PyQT. And I will keep the more simple, ugly interface around too. :-)

Posted: **Fri Mar 07, 2014 08:46**

Script updated: Now it also works with Python 3.

Posted: **Fri Mar 07, 2014 12:37**

Morn76 wrote:Yes, but GTK would be another dependency, while Tkinter comes with Python. Linux has advanced package managers like pacman, so installing dependencies is not a problem, but on Windows and OS X they are quite annoying. And this thing is more geared towards Windows. Because on Linux I can just navigate to a mod folder with Dolphin, press F4, and type 'git pull'.

I may do an upgraded version, but I think I would be using PyQT. And I will keep the more simple, ugly interface around too. :-)

That's a valid reason. I guess pygtk is just a personal preference.
EDIT: Actually, I'm learning to use pygtk so I might actually try converting it ...(maybe)

Posted: **Fri Mar 07, 2014 12:48**

Evergreen wrote:That's a valid reason. I guess pygtk is just a personal preference.
EDIT: Actually, I'm learning to use pygtk so I might actually try converting it ...(maybe)

I've looked at PyGTK code and found it not very Pythonic. It's just a minimal wrapper for the C library. More complicated than Tkinter and not necessarily hugely more powerful, unless you need some modern widgets that Tk's basic 1980s-style widget types do not offer. But for basic widgets like buttons, entries, text boxes, etc. it's not a very big improvement IMO, neither in the coding nor in the looks department.

I mainly use KDE on Linux, so with Qt, I could integrate this tool into the KDE task bar so it could e.g. check for updates once daily. And you draw the GUI with Qt Designer I think, no coding necessary. Now that is what I call a modern GUI toolkit. :-)

P.S. Of course I'm not trying to dissuade you from porting to GTK if you want to. I don't hate GTK when others do the work for me. ;-)

P.P.S. Another toolkit possibility is wxWidgets. I think it uses GTK on Linux and native widgets on Windows. I've tried this out once in a project because GTK looked a bit too convoluted and it wasn't so bad actually.

Posted: **Fri Mar 07, 2014 13:26**

Neat :)
Could it be done so, that one could change the repo?
Sometimes authors discontinue their stuff and a new modder takes over within his/her own repo.
*cough* search for mods on githug would be great *cough*

Posted: **Fri Mar 07, 2014 15:37**

Useful tool.
Maybe could you add a "small database", so it gets the URL of the GutHub repo automatically?
With this way, I would need to search the creator / forum thread first.

Posted: **Fri Mar 07, 2014 15:41**

You could use the mmdb API, or you could search github.

Posted: **Fri Mar 07, 2014 17:44**

Those are very good ideas. The built-in Minetest modstore never has anything I want or need, so a decent list of the best forks for each mod might be handy.

Also, I think the tool should be smarter because the GitHub repo name is sometimes different from the mod folder name. If the GH repo is called minetest-foo but the correct folder name is mods/foo the tool should rename things accordingly.

Posted: **Fri Mar 07, 2014 20:57**

I have added a simple mod database to the tool (under "Browse mods") in v3.0. Click "Select", then "Add" in the main window.

The current list is from the Stampy world map thread, so it is not very complete yet. Suggestions for inclusion are welcome.

Posted: **Sat Mar 08, 2014 08:29**

New version (v3.5) now with more included mods in the database

Posted: **Sat Mar 08, 2014 11:43**

4aiman wrote:Neat :)
Could it be done so, that one could change the repo?
Sometimes authors discontinue their stuff and a new modder takes over within his/her own repo.

I think in that case it would be better to delete the existing repo folder e.g. from your file manager, then clone the repo from the new maintainer URL.

It is possible to redefine the upstream URL for a repository ("git remote set-url origin"), but I really want to keep this program simple and not introduce anything that might mess with existing git repos in the mods folder. So I want to limit it to "git clone" and "git pull" essentially, because those two are pretty safe.

Posted: **Sat Mar 08, 2014 11:52**

Wow, nice!

This is already a lot more useful than MMDB.

It would be cool to a have a list of the mods that are in your mod directory and an option to just update a single one. I usually have my indev mods there too, and a merge would mess them up.

Posted: **Sat Mar 08, 2014 15:02**

PilzAdam wrote:It would be cool to a have a list of the mods that are in your mod directory and an option to just update a single one. I usually have my indev mods there too, and a merge would mess them up.

Maybe the program could check each directory for a file that serves as a flag, e.g. "._MTGITSYNC_IGNORE_" and a directory that contains it would get skipped during updates. Because e.g. showing a scrollable list of installed mods with checkboxes in front of them would be very difficult or even impossible with Tk widgets. Most people will probably just want to update everything, so thís could be more of an advanced, slightly hidden feature for mod devs only.

Posted: **Sat Mar 08, 2014 15:59**

New in version v3.6:

"Web page" button which opens the GitHub page for a mod in your web browser. (Which in most cases has a link to a forum thread or at least a decent README.)
A flag to skip some directories during updates, see the green text in the first post for a description. This is mainly intended for mod devs.

Posted: **Tue Mar 11, 2014 10:21**

Version v3.8 has some nice improvements compared to v3.6, e.g. spaces in folder names are no longer a problem. (Normally GitHub repos should not contain spaces, but you never know. And I suppose this might have affected Windows users who keep Minetest in e.g. "My Documents".)

And the official license is CC0, which is basically WTFPL without the swear words. :-)

I think this program now has pretty much all the functionality you can expect from a Tk app; anything beyond that (e.g. a built-in GitHub browser or whatever) would require a more powerful widget set like Wx, GTK, or Qt.

Posted: **Sun Apr 27, 2014 09:31**

Great app, wish I'd seen it months ago!
oddly under ubuntu 14.04 tkinter isn't installed by default, so you have install python-tk (for 2.7), or python3-tk from the repository (then it works fine)
Just clicking in the path, typing the mods directory path and clicking out didn't update the path (it was still looking for /home/martin/...), but this was only apparent when looking in the terminal, the error didn't appear in the application window.
I must have been a little dense, but I also spent a minute clicking "Update now" before realising that wasn't actually the button which added the current mod. Perhaps making the addmod more obvious and renaming the update to "Update All"
Also if you try to clone a GH but the folder already exists, the old 'temp' folder is left behind in the MT-GitSync dir, and causes subsequent errors even if the original folder is deleted (ie the script should delete the temp folder on error, or display a message to enable overwrite).
Also is it possible on the next version to have the browse listbox resizable?
Also trying to think if it would be helpful or not to mention in your user instructions that you can edit the modtable in the py script if you want to change the output folder name (eg if the mod is set up incorrectly).
Tx

Posted: **Sun Apr 27, 2014 20:19**

Thanks for your suggestions, dgm5555! I've made a new version (3.10).

dgm5555 wrote:Great app, wish I'd seen it months ago!

It only exists since last month, so that would have been impossible. :-)

dgm5555 wrote:oddly under ubuntu 14.04 tkinter isn't installed by default, so you have install python-tk (for 2.7), or python3-tk from the repository (then it works fine)

Noted now in README and forum post.

dgm5555 wrote:Just clicking in the path, typing the mods directory path and clicking out didn't update the path (it was still looking for /home/martin/...), but this was only apparent when looking in the terminal, the error didn't appear in the application window.

Made it so the text box can no longer be selected/typed in. I agree it was not very consistent because the mod name box can be typed in and will use what's inside the text box.

dgm5555 wrote:I must have been a little dense, but I also spent a minute clicking "Update now" before realising that wasn't actually the button which added the current mod. Perhaps making the addmod more obvious and renaming the update to "Update All"

Renamed the buttons a bit.

dgm5555 wrote:Also if you try to clone a GH but the folder already exists, the old 'temp' folder is left behind in the MT-GitSync dir, and causes subsequent errors even if the original folder is deleted (ie the script should delete the temp folder on error, or display a message to enable overwrite).

This is kind of by purpose. Following the KISS philosophy, I don't want this script to remove anything, ever. That is up to the user. Even if the script *thinks* it is just removing a useless folder, it might not be. So failing noisily on bad moves where e.g. the target already exists is a good thing IMO.

dgm5555 wrote:Also is it possible on the next version to have the browse listbox resizable?

Resizable is difficult in Tkinter, so I've increased the number of visible lines from the default of 10 to 25.

dgm5555 wrote:Also trying to think if it would be helpful or not to mention in your user instructions that you can edit the modtable in the py script if you want to change the output folder name (eg if the mod is set up incorrectly).
Tx

I've put this in the README too. I suppose the list of mods could easily be made an external file too, e.g. a csv-formatted text file. It was just for my convenience that I wanted to minimize the number of files. But in princriple I know that separating code and data is a good idea. :-)

Posted: **Mon Apr 28, 2014 13:32**

P.S. I've moved the mod table to a .csv file for easier editing in 3.11. Forks with updates to the .csv are welcome. Hint, hint. :-)

By the way, the .csv does not have to be sorted alphabetically (because the script sorts it anyway after reading). But maybe it makes it a bit easier to see which mods are already included.

Posted: **Mon Apr 28, 2014 21:51**

OK my next request then is that the tool accesses an online page which stores the names of the mods and can save names of 'unknown' mods, thus everyone can contribute, and if it was really smart, could track rates of downloads. Perhaps it could use/modify the modstore list from the MT menu (though I find MT-gitsync tidier)...

Also without some sort of auto-merging/listing of unknown mods, a list like the following is impossible to figure out what's new or known, and takes ages to manually compare lists (probably why you don't get many contributors):-
minermoder27/carts
Adding minermoder27/carts... cloned... moved... done!
Adding rubenwardy/food... cloned... moved... done!
Adding Zeg9/minetest-itemframes... cloned... renamed... moved... done!
Adding mdoege/minetest-hatches... cloned... moved... done!
Adding Jeija/minetest-mod-mesecons... cloned... moved... done!
Adding bdjnk/mini_sun... cloned... moved... done!
Adding Neuromancer56/phonics... cloned... moved... done!
Adding Sokomine/locks... cloned... moved... done!
Adding Sokomine/mobf_animals... cloned... moved... done!

Also can MT-GitSync automatically update itself? The MT-GitSync folder doesn't seem to be listed when I click update all despite being in the mods folder.

Posted: **Mon Apr 28, 2014 22:51**

dgm5555 wrote:OK my next request then is that the tool accesses an online page which stores the names of the mods and can save names of 'unknown' mods, thus everyone can contribute, and if it was really smart, could track rates of downloads. Perhaps it could use/modify the modstore list from the MT menu (though I find MT-gitsync tidier)...

There is also the Minetest mod database ( https://forum.minetest.net/mmdb/ ), which has been suggested as a data source for MT-GitSync earlier in the thread. But the thing is, they only have four pages of mods, so I think it's hardly worth the effort to integrate this with MT-GitSync. In fact neither the modstore nor the mod database seem to get a lot of updates or attention.

dgm5555 wrote:Also without some sort of auto-merging/listing of unknown mods, a list like the following is impossible to figure out what's new or known, and takes ages to manually compare lists (probably why you don't get many contributors)

True, then again the built-in mod list was never intended to be an exhaustive list of every single Minetest mod on GitHub. There are so many, especially if you count every fork. That list would be impossible to maintain, even in a crowdsourced fashion.

Instead the list was supposed to help new MT users find some commonly installed and popular mods. Especially also for features that are standard in MC but not implemented in minetest_game: carts, mobs, TNT, hunger, mesecons, beds, arrows, those kinds of things. I don't want to clutter the list too much with obscure mods that only a few users are interested in.

dgm5555 wrote:Also can MT-GitSync automatically update itself? The MT-GitSync folder doesn't seem to be listed when I click update all despite being in the mods folder.

My assumption was that many users will get MT-GitSync as a ZIP file, which of course also makes it harder to update itself. Just pulling random raw files from GitHub could be flaky and is generally not a good idea for code that is executed.

Maybe MT-GitSync could check if it is inside its own git repo and if it's not, check itself out into a subdirectory. And then if you run the git repo version, it could do git pull. But would users understand that? And just removing files from the working directory or something like that is dangerous, see my earlier comments about removing files or directories. So basically a full auto-update without git makes me a bit queasy.

Perhaps the way to go would be to pull automatic updates for modlist.csv from GitHub (that's pretty safe, especially if there is error handling in the script for a corrupted modlist). For the main program, maybe just do a version check against GitHub and inform the user when a new version is available.

Posted: **Tue Apr 29, 2014 00:01**

Morn76 wrote:In fact neither the modstore nor the mod database seem to get a lot of updates or attention.

It would be great if more modders could add their mods there. Installing from the mmdb is pretty easy.

Posted: **Tue Apr 29, 2014 09:59**

Sokomine wrote:
Morn76 wrote:In fact neither the modstore nor the mod database seem to get a lot of updates or attention.

It would be great if more modders could add their mods there. Installing from the mmdb is pretty easy.

Yes, installation is easy, but you never get updates, do you? Mmdb mods are distributed as ZIP files.

Posted: **Tue Apr 29, 2014 19:14**

Morn76 wrote:Mmdb mods are distributed as ZIP files.

zips irritates me too, one of the reasons I like MT-GitSync

Posted: **Tue Apr 29, 2014 19:42**

dgm5555 wrote:
Morn76 wrote:Mmdb mods are distributed as ZIP files.

zips irritates me too, one of the reasons I like MT-GitSync

And that's also why I think the approaches of mmdb and MT-GitSync are fundamentally incompatible. Pretty much all mods in active development are on GitHub already and those which are not in active development will probably stop working after the next API change. So why not use GitHub for distribution?

Also, there are situations where mods interfere with each other, but at least you can only hope for an update to fix that. E.g. I'm still waiting for 3d_armor and wieldview to work together properly. :-) So I think updates are as important for mods as for the main game.

Posted: **Tue Apr 29, 2014 20:36**

Morn76 wrote:
Sokomine wrote:
Morn76 wrote:In fact neither the modstore nor the mod database seem to get a lot of updates or attention.

It would be great if more modders could add their mods there. Installing from the mmdb is pretty easy.

Yes, installation is easy, but you never get updates, do you? Mmdb mods are distributed as ZIP files.

I do use it :D My StreetsMod just waits for an update :P

Posted: **Tue Apr 29, 2014 21:24**

webdesigner97 wrote:I do use it :D My StreetsMod just waits for an update :P

Great, but how are users going to get that update without downloading a new ZIP file? Not very convenient if you ask me.

FOSS gamedev and creative worlds

[Tool] GUI for installing and updating mods from GitHub

[Tool] GUI for installing and updating mods from GitHub

Re: [Tool] Python GUI for installing and updating mods from

Re: [Tool] Python GUI for installing and updating mods from

Re: [Tool] GUI for installing and updating mods from GitHub

Re: [Tool] GUI for installing and updating mods from GitHub

Re: [Tool] GUI for installing and updating mods from GitHub

Re: [Tool] GUI for installing and updating mods from GitHub

Re: [Tool] GUI for installing and updating mods from GitHub

Re: [Tool] GUI for installing and updating mods from GitHub

Re: [Tool] GUI for installing and updating mods from GitHub

Re: [Tool] GUI for installing and updating mods from GitHub

Re: [Tool] GUI for installing and updating mods from GitHub