Modular Rendering System

From ioquake3 wiki
Jump to: navigation, search

This acticle is out of date and doesn't reflect current state of ioquake3. ioquake3 now has a working modular renderer system and many things might not be planned to be done.

Modular Rendering System

ioquake3's goal is to modernize the Quake 3 engine while maintaining compatibility with Quake 3 1.32 media, servers, clients, and general hardware requirements. However, the Quake 3 engine is over 10 years old and while the gameplay and networking are still relevent, the graphics and content capabilies are not.

This presents several problems for people and projects wishing to use the ioquake3 engine:

  1. Content creators (mappers, modelers, animators, texture artists) typically want to use "current" (Doom 3 level, 7 years old) technology
  2. mod/game projects attempting to attract free contributors have a hard time because no one wants to work on old tools and techniques that won't help them to get a job in "the industry"
  3. Modern graphic hardware allows offloading some of the rendering duties from the CPU but ioquake3 doesn't support this which makes performance even worse for anyone trying to use ioquake3 on a mobile device
  4. Modern graphic hardware allows far higher performance which in turn allows for more detailed (geometry-wise) environments/maps but ioquake3 forces everything through the CPU and you hit performance limits with scenes involving 60,000 triangles on hardware that can easily do scenes of 1,000,000 to 10,000,000 triangles

One of the perceived problems with modernizing the rendering system to allow for "modern" effects is that it would break compatibility with Quake 3 1.32 content and binaries. This issue can be mostly mitigated through careful design. The following represents an attempt to approach this issue by allowing for better performance and eyecandy while minimizing the impact to those unable to use anything better than an UltraSPARC.


Modular Part

Ideally, the current ioquake3 rendering system would be isolated as much as possible to facilitate "replacement" or being swapped for another code path. As I am not familiar with the ioquake3 codebase, this represents an idealized high-level view and it should be revised by individuals with actual working knowledge of the engine as far as feasibility of implementation goes.

Currently, ioquake3 has the traditional renderer, a tweak by Thilo to allow for anaglyph stereo rendering (3D glasses), and an optional (and possibly outdated) cell-shading renderer as a patch. I would think that Thilo's anaglyph stereo rendering ability would ideally be a separate function from any of the "renderers" as who is to say you wouldn't want the cell-shading renderer in "3D"? If it is possible to separate that out to be a base ability in the ioquake3 engine, that leaves two renderers currently associated with ioquake3, the traditional one and the cell-shading one. These should be used as test cases to determine the feasibility and functionality of a modular rendering system for use with ioquake3.

As one of the goals for implementing this system is to allow for taking advantage of the newer hardware, after implementation of those two renderers, XreaL's renderer should then be attempted to be integrated. In fact, XreaL may be a good starting place to examine as Tr3B apparently modularized the renderer a bit already. Tr3B has already publicly stated that he does not mind if anyone uses any of the GPL'ed code from XreaL which also makes this an ideal test candidate. This may also be a better stepping stone for OpenGL ES implementations for coders attempting to do mobile ports of ioquake3.

Quake 2 had such a modular rendering system though I believe the rendering was split out into platform-specific binaries. If you wanted the PowerVR path, for example, it would load powervr.dll instead of opengl.dll. I do not know if a similar system would have to be employed for modularizing ioquake3's rendering but at least there is precedence. I suppose, too, that it would make it easier to pop in different renderers to try without recompiling from source every time. I wonder if you could abuse the QVM system somehow to make drop-in renderers that work on any platform?

Compatibility

The main concern is compatibilty with older content and binaries, the fear that having maps that take advantage of a newer renderer's allowed effects will cause 10-year-old Quake 3 clients to crash or have problems. There are several cases to consider and hopefully they are adequately covered. A new cvar should be added, perhaps something like cl_renderer or r_renderer, which would be an 8-bit value. The low values (1 to 127) would indicate ioquake3-included renderers and the high values (128-255) would indicate community-provided renderers.

As an example, 1 would be for the traditional ioquake3 renderer, 2 would be for the XreaL renderer, 3 would be for the cell-shading renderer, and perhaps 128 would be for an OpenGL ES renderer that someone has contributed as a patch or that ioquake3 is aware of but that is not actually included in the ioquake3 source tree. In this way the community can continue to work on different renderers and inform ioquake3 without ioquake3 feeling obligated to include the work in the main source but can provide a mechanism for the community at large to ensure content gets used with the intended renderer.

You could even include a separate file to be maintained independently of releases, say iorenderers.ini, which would define the value assigned to the renderer and the name. This might even be something that the ioquake3 master server could host, perhaps on server startup or once a day or week or on map changes/loads the ioquake3 server could query the master server (or a defined URL in a .cfg) for a newer version of the iorenderers.ini file and automatically update it, if the server op chooses to enable this ability. This would allow content and renderers to grow independent of ioquake3 release schedules. Depending on how many renderers eventually pop up, you could even leave the high values the same for "legacy" compatibility even after you move some of those renderers to the lower values due to being officially included in the ioquake3 source tree. I suppose then one may ask "why differentiate to begin with?" but I would assume that any point releases for ioquake3 would entail cleaning up this iorenderers.ini file, perhaps limiting updates only for the most recent point release. i.e. ioquake3 1.38 would have updates, 1.39 would bring a cleanup and existing servers would no longer get iorenderers.ini updates until they update their binaries. The server would be smart enough to read the first line which might be something like "version.1.39.100505.2105" [version.1.39.YYMMDD.HHMM] and realize that the "1.39" doesn't match its current version of 1.38 so therefore don't check the rest of the version string and don't update at all.

A mapper creating a map typically generates an .arena file that defines the allowed gametypes to be used with the map. In the same way they can also provide a line in the .arena file (as long as that won't cause problems when Quake 3 1.32 parses it) or maybe a separate .renderer file to indicate which renderers the map can be used with. The client parsing the map will therefore be able to exclude a certain map or indicate to the user via feedback in the console that they have a map that is currently being filtered out due to renderer restrictions. This will only work on ioquake3. For a Quake 3 1.32 client, the map will just fail in some magical way. In this case, the mapper will need to have provided information as to what version of "Quake 3" the map can be used with. Perhaps this can be done with a secondary levelshot/preview. With the "normal" one, maybe make an ioquake3-standard placeholder that any mapper can use such as text that says, "MAP CAN ONLY BE USED WITH IOQUAKE3". Then, make a secondary levelshot that has the "real" preview. ioquake3 will look for that levelshot specifically to use and, if present, ignore the "normal" one. In fact, one could do this with the .arena/.renderer method as well... the .renderer would have the same info as the .arena file with the addition of what renderers are allowed via the renderer cvar value and the .arena file would be blank and allow for nothing. If it holds more than just the renderer support info, then, perhaps a better extension can be used such as .ioarena instead.

On the server side the same considerations can be made. However, I am unfamiliar with server map parsing code so I do not know if the server actually cares about .arena files or if they are only used by clients when choosing maps via the UI. If the server parses .arena files, the same general solution may work as on the client with an additional entry or perhaps a "blank" .arena to prevent the map from being marked as usable on non-ioquake3 servers with the information instead in a .render or .ioarena file. The server code should snoop clients for the cl_renderer or r_renderer cvar and compare that with the currently-loaded map. If the client's renderer is not supported, either kick the client with an error message stating what's wrong (map requires different renderer, please switch to X instead) or if the cvar is not found (indicating a non-ioquake3 client) and the basic renderer is not listed as supported by the current map, kick the client with an error message stating what's wrong (map requires ioquake3 running X renderer, please go to www.ioquake3.org and download the client). It may be desirable, depending on how a map fails to display on other Quake 3 clients, to allow the client to connect and load and instead abuse/reuse the MOTD code to pop up a message on the screen (a few seconds after the existing server MOTD, if there is one) to provide the same information and how to resolve the map that looks bad/doesn't work right. You could even include support on a per-renderer basis via the iorenderers.ini file as to what behavior you want, give control to the mapper via the .render/.ioarena file, or let the server op decide via another server cvar. Or multiple levels, each with a certain override of the others, say iorenderers.ini as a baseline, .render/.ioarena overrides that, and server cvar overrides both as the server op should have the ultimate option to dictate the playing experience on the server they are running.

Enhanced Model/Animation Support

One of the desirable bullet points for attracting artists is the ability to use a more "modern" model and animation format. In this case, exporting to vertex animation is old and busted but exporting to skeletal animation is the "new" hotness. Being able to take advantage of things like normal maps are also of interest to modelers as making a normal map from a high-poly model to apply to a lower-poly version of that same model is part of the typical workflow that is used in the gaming industry nowadays. Being able to offer that same workflow and having real results is far more attractive than telling your potential artists, "well, do it like you'd like to do and then we'll just ignore all the work when we dump it into an MD3." Yes, the artist can still do the same workflow but without the payoff of seeing it used in-game in all its glory, the artist tends to get unhappy.

The MD5 format support in XreaL is a great starting point. However, I do not know if this format can gracefully be degraded for legacy renderer support. If so, then adding support for MD5 should be "global" for the engine in the sense that it would be available to any renderer.

Compatibility may be harder to pull off. I do not know if animation.cfg can be abused in the same manner as the .arena file to attempt to block the load of a MD5 by Quake 3 1.32 clients. However, being able to "import" fancier content created for Doom 3 and beyond will help "sell" ioquake3 to new players by showing off more eyecandy for marketing materials like videos and screenshots. This also has a secondary effect of helping any project using the ioquake3 market their own project with available improved eyecandy.

TL;DR Version

  1. Separate out the rendering code, maybe put each renderer in a different platform-specific library like Quake 2
  2. If anaglyph 3D can be separated out from being renderer-specific, it should be globally available for all renderers
  3. ioquake3 maintains master list that servers can get updates with, something like iorenderers.ini with version control
  4. ioquake3 server periodically queries for a new version of iorenderers.ini and updates local copy as needed
  5. Add new cvar like r_renderer or cl_renderer for the client to control what renderer to use
  6. Mappers using new renderer features add a line in their .arena file to specify which renderer(s) the map will work with
  7. For backward compatibility, maybe use a .renderer or .ioarena file with that info and make the .arena file blank so that other Q3 clients ignore the map as being available to be used/loaded with any gametypes
  8. Backward compatibility would also be helped with a secondary levelshot; the "normal" one would display a placeholder image saying to switch to the ioquake3 client and the secondary one would be used by ioquake3 automatically
  9. Server snoops client's renderer cvar to check against currently-loaded map's requested/compatible renderer
  10. Four methods for handling render mismatches; kick the client with an error message for wrong renderer, kick the client with an error message for wrong client (Q3 vs ioq3), let the client load and stuff MOTD about the wrong renderer, let the client load and stuff MOTD about wring client (Q3 vs ioq3)
  11. Added bonus, while a Q3 client is loading, if the secondary levelshot method is used, client will also get the placeholder levelshot about needing to use ioq3 with a specific renderer
  12. Possibility to add finegrained control for renderer checking, let kick versus load depend on iorenderers.ini per-render preference, .arena and .renderer/.ioarena per-map per-renderer preference, and per-server preference depending on the wishes of the server op
  13. Server op preference overrides per-map and pre-renderer, per-map overrides per-renderer
  14. MD5 support should be added to help attract content creators
  15. If MD5 support can be degraded gracefully depending on renderer, it should be globally available for all renderers
  16. MD5 backward compatibility may be harder to pull off depending on how much we can abuse animation.cfg or whatever files Q3 loads for model support