wiki.libsdl.org migration...

Heads up, we’re moving to our new wiki today.

tl;dr: the wiki looks different but mostly works like a wiki and you can ignore this unless it affects you personally.

THIS IS A WORK IN PROGRESS and if something feels wrong, it’s either a bug to be fixed or something we’d like to hear about so we can improve.

I’m just going to explain why this is happening, as change is always scary.

These were the problems we were trying to solve:

  • MoinMoin is kinda old and crusty.
  • Everything else moved to GitHub, which offers a wiki, so if we were dumping other old and crusty services for GitHub equivalents, moving the wiki made sense, too.

But almost immediately, we discovered we had a new problem:

  • GitHub’s built-in wiki sucks, too.

So we stopped for a bit to think about what we’d like the perfect wiki to do, and here’s some things we came up with:

  • We liked the idea of the wiki being backed by a git repo, so wiki users can use a web interface, but there’s nothing stopping us from iterating on the content with a perl script locally, too.
  • We have a separate Git repo for the wiki contents (“sdlwiki”) so we don’t get a ton of update spam in the main SDL repo, since wiki commits tend to be smaller and more frequent than source code commits (there was 7000+ commits in the MoinMoin data set).
  • We publish the contents of this repo at wiki.libsdl.org with some other bits of user interface.
  • As changes are pushed to this repo, we get a notification from a GitHub webhook, like we do to tweet and post to Discourse. This notification causes wiki.libsdl.org to pull the latest and generate static HTML for changed files.
  • Each page has an “edit” button as one would expect from a Wiki. Clicking the edit button pushes the user through GitHub OAuth, which will tell wiki.libsdl.org their valid GitHub login name (and let them optionally create a GitHub account, etc). This fixes a massive problem we had with MoinMoin: spammers. They have to have a valid GitHub account, pushing spam detection onto GitHub, and causing a barrier to entry for spammers.
  • Submitting an edit causes wiki.libsdl.org to make a topic branch in a local clone of the sdlwiki repo, listed as authored by that GitHub user, with the changed file as the only commit. It then pushes this to GitHub and generates a Pull Request for it with the GitHub API. User is redirected to this pull request. They’re listed as the author on the commit in the PR, but the actual PR will come from a bot ("@SDLWikiBot").
  • We humans go through pull requests and approve/merge them like any other. When merged, wiki.libsdl.org starts publishing the changes immediately at this point because it got a webhook notification.
  • Extra credit: if we like the user, we mark them as trusted, and their edits stop making pull requests and just push the changes right to the main sdlwiki repo, reducing friction.

Benefits:

  • GitHub manages accounts, deals with spammers, etc.
  • We can pipe this stuff through pull requests, keep a revision history, etc.
  • Contributors don’t have to fork the repo or worry about keeping it up to date. They just edit in a webpage, replacing the latest contents.
  • If someone wants to make a pull request by cloning the repo (they want to run a perl script over a bunch of files, etc), they can still do that in the usual GitHub ways.
  • If two people edit the same page, they’re in separate pull requests and GitHub will tell us we need to fix it up if it can’t automatically merge, but that’s something we can handle on our end.
  • If someone is obnoxious or a spammer, we can block them on the wiki and then they can’t submit further pull requests (but they will still have access to read the wiki).
  • We can offer zip files of offline HTML, since we control the software.
  • Editors can interchangeably choose MediaWiki or GitHub-flavored Markdown for any given page.

So I took a few weeks to write this, and that lives at the ghwikipp project page.

One more benefit: Edits to the SDL headers and the wiki are now bridged, so changes to one will show up in the other place automatically. That code is sitting in SDL/build-scripts/wikiheaders.pl, and runs in a cronjob on libsdl.org.

This is now live on https://wiki.libsdl.org/ … This is still early times, so some MoinMoin content hasn’t been cleaned out, and there are lots of CSS tweaks and little changes we would like to make still.

But we are happy with this as a first cut and intend to continue to improve it, and appreciate your feedback and questions here.

Thanks!

3 Likes

Is the old wiki still viewable in a read-only state anywhere? It’d be handy for trying to bring stuff up to snuff in terms of tables and such which used to have a certain type of formatting? Just to use as a reference for updating things.

We worked with ArchiveTeam to make sure the whole thing was on the Wayback Machine.

https://web.archive.org/web/20210210110800/https://wiki.libsdl.org/

2 Likes

Bug: For some reason CategoryEvents - SDL Wiki doesn’t exist, even though sdlwiki/CategoryEvents.mediawiki at main · libsdl-org/sdlwiki · GitHub does

EDIT: should’ve created a github issue… here it is: CategoryEvents broken · Issue #31 · icculus/ghwikipp · GitHub

Hi,

I appreciate that you provide a better way to collaborate to the wiki. But I still don’t understand why you don’t want use doxygen as the SDL headers are already written using its syntax. If it’s because of its ugly output you can already use other software using its same syntax (e.g. sphinx can be bridged to it through Breathe, with that we could write the wiki along with the sphinx doc as well too). At the moment you must write the doc two times and sync it two times each time you make a modification. Also, I spent literally hours to document the SDL sensors API by editing the wiki myself. It’s a terrible huge waste of time.

I think it’s time to ditch the wiki for everything related to the API and keep it only for things that are outside the scope of the API (articles, platforms notes, etc).

Also, I spent literally hours to document the SDL sensors API by editing the wiki myself.

If these were lost in the transition (sorry!), we can pull them out of revision control and get them into the appropriate place. Right now changes to the headers bridge to the wiki automatically, but (until I’m more confident in the software), we don’t automatically merge the other direction. Eventually, changes to either place will automatically propagate to the other.

The headers do work with Doxygen, and a lot of effort is spent making sure the wiki contents magically convert into a Doxygen-compatible format, so people that want to use it can have it.

We have a lot of tweaking to do still, but I’m not unhappy with how the wiki is turning out so far.

If these were lost in the transition (sorry!), we can pull them out of revision control and get them into the appropriate place

No I documented it a while back ago, can’t remember but at least one year old. It’s definitely present right now.

Right now changes to the headers bridge to the wiki automatically, but (until I’m more confident in the software), we don’t automatically merge the other direction. Eventually, changes to either place will automatically propagate to the other.

Good to hear! In the other direction do you mean that the wiki should be able to merge into the .h files? That’s sound quite complex if I understand correctly. If that’s the case I think it would be better if the headers/git repository from SDL stays the source of truth (and so the API wiki stays read-only mirror). And as such, fixing or improving the documentation regarding the API should be done by sending PRs right into the SDL tree.

It isn’t simple, but we’re talking a few hundred lines of perl, not thousands. :slight_smile:

When trusted users edit the wiki, changes go directly in (and, when I turn the system on, will commit to the headers at some interval). Untrusted users can edit the wiki, but their edits automatically become pull requests…both for quality control purposes and to prevent them making changes that would be damaging to the headers without oversight.

Since it’s bridged, this is a perfectly viable way to update everything at once. But there are a lot of people that just want to click “edit,” make a small change, and click “submit” and never think about it again, and the wiki interface is super useful for that. We took care of all the juggling to make sure this Just Works for everyone.