CI/CD For Morrowind Modders

Posted:

It may seem like an odd intersection but leveraging CI/CD patterns can vastly improve both the developer and user experience if you're a content creator such as a mod-maker. Morrowind (my favorite video game, I'd say) has a very active modding community even 21+ years after its release. Read on to see how I use CI/CD to enhance mine and my users' experience and to learn a bit about the Morrowind modding process in general!

→ What Is Morrowind?

The Elder Scrolls III: Morrowind is a PC role-playing game released in 2002, made by Bethesda Softworks. It's an "open world" game in that you can go anywere you can see and interact with basically everything. You can even do things to break the intended quest flow. To say they don't make games like this anymore is true I think; modern Bethesda games do resemble Morrowind in various ways, but they have also been designed for a very different audience. Anyways, I consider it a timeless classic even with its aged quirkiness and flaws.

→ What Is Modding?

Generally speaking: modding is the process of altering games. Some games are easier to alter than others because they were designed in such a way as to encourage it. Morrowind was in fact designed in such a way and even shipped with "The Construction Set" which is essentially the same software that the designers used to make the game content, such as quests and other records found in the game data. This means that prospective modders have very powerful tools at their disposal and can effectively extend the game indefinitely.

→ What Is CI/CD?

CI/CD is one of those terms that may mean different things to different people, but in this context I mean: continuous integration and continuous delivery. I want to have constant feedback about my stuff and I want to have easy access to work-in-progress builds for my users (let alone releases). Most of my mods lack tests so you could argue there's no real "CI" going on, but I do consider providing easy access to non-release builds as a kind of continuous integration; the most practical way to test mods (at present, at least) is to simply use them.

→ The Workflow

Diving into the details a bit more:

For folks that have worked in software development, none of this should be too unfamiliar. But let's take a look at each point in detail and see how it works in the context of Morrowind modding!

→ Using A Code Forge

There are several options out there for code forges, use what you like best. Note that not all code forges have all the features I'm using, so keep that in mind. My full workflow involves these services:

Each of these will be explained further, but in any case while your code forge need not have all of these features it will make things a bit nicer for you. Especially for artifact and image hosting, which are a bit of an investment to self-manage.

→ Modders' Workflow

Back to the modding process: there's effectively two kinds of mods for Morrowind: mods that replace some asset like a mesh or texture, and mods that add content like quests, landmasses, cities, etc. The asset variety of mod fits less well into my model since it involves work on potentially large binary files, but it can still work just fine. My personal modding focus is on the content variety; that is: creating plugins to add or alter in-game content like quests, item placement, and so forth.

The typical workflow for this would be to fire up the Construction Set and get to work. I personally use OpenMW-CS for all my heavy lifting but the high-level flow is the same. Add or alter records, scripts, scenes, and what have you. Save your work and OpenMW-CS saves a .omwaddon file that's mostly equivalent to the .ESP files the vanilla CS executable spits out. The vanilla engine uses compiled scripts, whereas OpenMW's scripting is interpreted and there's no compilation going on. So if you make a mod with scripts using OpenMW-CS, it won't work right in the vanilla exe because scripts won't be compiled.

Stuff it into a zip of some kind and this is where most projects call it a release, but I take things a little bit further.

→ Version Control

While working it's natural to save as you go. That's all well and good, but I need more than just hitting Ctrl-s in the CS and getting a new .omwaddon. I want to save my progress using a version control system: git. All scripts and files in general should be versioned and keep a history. But committing the binary .omwaddon seems a little nasty, and we can do better.

→ Plugins As Text

Enter: DeltaPlugin by Benjamin Winger, which can serialize the binary plugin to YAML-formatted text. With this I can much more easily visualize changes to my plugins since all changes are recorded as text. In order to do its thing, DeltaPlugin reads an openmw.cfg and scans the plugin loadout. Each plugin's data is checked for information about "masters" (dependencies) and those are read too, as needed.

In this case where I'm converting from plugin to text and back, DeltaPlugin reads data in the YAML file to determine if any masters are needed and ensures that they are there too. But this poses a problem for my aspirations of building via CI: Morrowind mods naturally depend on the Morrowind content files, which are proprietary. I definitely can't be using those in CI, so what else can I do?

→ MockMW

To solve this problem I've created a set of empty plugins with DeltaPlugin that have the same file name as the proprietary plugins but without any of the content, I call it MockMW and it gets shipped as a Docker image. For the purposes of building in CI this is more than good enough!

The MockMW repo contains several YAML files for various proprietary content files as well as a build script for converting them to their expected filenames. I pack that all into a Docker image via CI which I then use in CI jobs for my other projects or even locally of course.

There are countless Morrowind plugins out there and it's impossible to cover everything, but right now MockMW supports many popular mods and extending it is pretty simple. With all that in place I can now commit often and have a CI-built plugin for my users to test on every commit.

→ Static Website

I could publish a release and direct users to that, but the fact is that the GitLab releases page (and likely the release page offered by any other code forge) is simply an information overload for non-programmers. For a software project that's consumed by tech-savvy individuals, these release pages are great, but I know from experience that non-technical users can be confused about all of the information, source links, and so forth, and are unsure about what they need to get.

This is why I've settled on using simple static websites to deliver content to users. These websites are designed to present only relevant information via a simple UI: mod title, latest version, download button. Below that, a button to download a "dev build", then some links, and further informational content. Take my mod Action Camera Swap as an example.

The big idea being: make it hard to get confused or do the wrong thing. Make the right thing the very obvious, happy path. I could get crazy with source links and other things that I personally would find interesting, but for video game related content less is more. Give people what they want (in this case, the mod itself), and make anything else easy enough to find for those that take the time to look.

Under the hood, the "Download" button uses a tiny bit of JavaScript to ask the GitLab API what this project's latest version (git tag, more on that below) and download artifact are and then update the version info and download link accordingly.

→ Releasing

In keeping with the spirit of "make things easy" the release process on my project is: put a git tag onto a commit, push it up. A CI pipeline will fire off which builds and packages the release version of the mod. Once the pipeline passes a new artifact it will be published to the package repository, and the static website's release text and link will be updated.

Nothing more to it. Right??

→ Nexus Mods

Of course, aside from me I assume the entirety of the rest of the modding community (for Morrowind, at least) rely on publishing files to Nexus mods, which has a long history with this particular community. Considering the services they offer for free, the site is an extremely valuable resource and I'm happily a paid subscriber. But if I could nitpick just a little, their UX leaves a little to be desired. The two main issues that bug me are:

  1. Manual form that's several pages long for uploading a new mod
  2. No way to search comments or issues; sometimes a mod has hundreds of comments and critical information may be buried several pages deep

But the fact is that virtually nobody will see my GitLab-hosted mods without some external promotion. People simply aren't looking there, nor is there really any way to find things if they were. If I put a mod on Nexus, it's a sure bet that I'll see at least a dozen downloads (probably a lot more) within a few days. It all depends on your content of course, but there's so much user engagement to be had on Nexus and that in and of itself is extremely valuable.

→ Conclusion

So what's the point of all this? Why not just zip things up by hand and just upload it to Nexus?

I'm not trying to change anyone's workflow but I think it would be nice to make this pattern easier for the uninitiated to adopt. Many of the benefits are agruably a gain for non-programmers too. In time I'd like to expand this workflow and smooth out the rough parts such that a modder who's never used git before could jump in.

For now it's just another intersection of my hobbies and work. I want to create mods and things of that nature, but as I do I can't help but use the patterns that work for me on "professional" projects. Feel free to join me on the weekends as I live stream these activities, and happy modding!

This page was last modified on: 2023-08-18