End of Google Summer of Code Project

I would just like to wrap up this blog with the end of the Google Summer of Code SDL2 meta-build system project.

Review of Project Goals

In review, my goals were as follows:

• Create a meta-build system that allows for simple generation of methods to build SDL2 on different platforms
• Replicate as much of the SDL2 build process as possible
• Keep the meta-build system portable and lightweight
• Make sure extending it as simple as can be, for future parts added to the project

The medium for meta-building I chose was Premake, which uses Lua as a configuration language for setting up projects in a functional way. Given the simplicity of both Lua and the premake API, it seemed ideal. Equally important, Premake also has the entire system packed into a single executable, rendering it completely portable.

Accomplishments

Given my goals, I feel I accomplished all of them in one way or another. The following is a list of accomplishments from my perspective (development perspective):

• The meta-build system provides a method of rapidly and conveniently generating new projects related to the SDL2 build tree (such as a new test or SDL2 library)
• Provides a cross-platform way to generate various SDL2 build methods using the same meta-build interface (everything runs through premake4.lua for MinGW, Windows, Linux, Mac OS X, and Xcode)
• It provides the possibility of as complex dependency checking as CMake or Autotools with cross-platform convenience of Premake (with some tweaking, of course)
• It provides tested capabilities of generating GNU Makefiles; Visual Studio C++ (2008, 2010, and 2012) project and solution files; and Xcode (3 and 4) project and workspace environments

Features Support (Final)

The Premake project supports the following:

• Cross-platform project and solution generation for various operating systems and build environments
• Build environments include GNU Make (MinGW/Cygwin/Linux); Xcode 3 and 4 (both OS X and iOS); and Visual Studio 2008, 2010, and 2012 (Windows)
• Supported operating systems include Windows, Mac OS X, and Linux, with side targets for MinGW and iOS
• Support for building the SDL2 library and the side SDL2main and SDL2test libraries
• Support for building all of the projects in the test suite implemented at the time of this project
• Support for building all of the iOS demo projects
• Partial Cygwin support
• Interface for implementing CMake and Autotool-esque dependency checking routines (slightly demonstrated in the SDL2 project targeted at Linux)

Post-Mortem

As a reflection, I've noted many things that have gone wrong with this project, or just varied significantly from the original vision. I had very little build experience coming into this project (created a few of my own premake setups, created a few makefiles in the past to my own need, used Visual Studio for a couple of years, and used Apache Ant in the past). My Autotools experienced was limited to running a configure script.

Nevertheless, despite the huge barriers I had to overcome in order to understand what was needed to adequately replicate the complex build setup of the SDL2 library, I rose to the challenge and had a lot of fun creating the project. I gained a huge amount of build system experience and I hope that my work helps SDL2 in its efforts and all those who use it.

The project ended up going well beyond what I originally expected, though some of my stretch goals were missed. I wanted to implement iOS support as a stretch, plus build all of the SDL2 extra libraries (like SDL2_mixer, etc.) The second one proved to be well out of scope and there was simply not enough time to implement them. iOS ended up being one of the last major hurdles, and I feel it was well worth it. I hoped to have setup a good environment if others wish to add support for those extra libraries in the future.

Beyond that, the project moved to Linux support and I ended up creating extra goals that were not originally in the project description. Linux, itself, was a stretch goal because I originally signed on with my mentors' expectations I was focusing on Visual Studio and Xcode. Those weren't enough for  me, though. After getting them working, I decided to turn my focus toward Linux and Cygwin. Dependency handling in Linux greatly challenged me, which is why I ended up developing the goal to replicate some sort of complex dependency checking system like CMake and Autotools have. I implemented a primitive one and demonstrated it worked. I hope others or myself can extend it in the future, perhaps even leading to it becoming part of Premake. As far as Cygwin support goes, I regret it ended up not happening. Cygwin currently has linker issues as is covered both in this blog and in the changelog I kept throughout the project.

One of the most unexpected parts of the project was how much I ended up committing to the Premake source code. I noted in the description of my project that Premake may lead to barriers in implementing a system as complex as SDL2 (especially with forward compatibility in mind), but how to approach solving the problems those barriers caused was the question. One could work around the code in hackish or unappealing ways (which was my original approach), or directly modify the Premake source code (which I ultimately decided to do later, when I reached insurpassable barriers). What was interesting was all of the changes I personally made to Premake ended up being committed to the stable branch of Premake, making the current Premake used in the SDL2 project one patch short of being the live version currently maintained (the one patch adding core iOS support).

Final Words

Overall, I really enjoyed my time on this project. I am very excited to see how the community reacts to this project and I can only hope for the best. This was a great experience and I would recommend Google Summer of Code to anyone dedicated and willing to have a ton of fun, meet new people, and learn a huge amount of information in a topic of their choice, with the plus of getting paid for it.

Signing out.

Ben Henning

Scrubbing Project Syntax & Complex Configurations

Greetings,

After a six day outing to South Dakota, I'm happy to return and get right back to work on SDL's meta-build system!

Cleaning up Project Syntax

Before, the project syntax was rather icky. It involved a lot of table manipulation, assignment, listing, you name it. I just didn't like it. I didn't plan on keeping it from the beginning. In fact, I wanted a syntax closer to premake's.

With premake, you just do a bunch of function calls in a specific order. However, it doesn't feel like you're making function calls. That's because in Lua, you don't need surrounding parentheses when making a function call if you're passing it one parameter and that parameter is either a table or a string. Conveniently, everything in premake takes either a string or a table. So, in all their examples, they demonstrate passing string literals and inline tables. This gives almost a functional programming feel (which is obviously the end goal on Lua's part). It's just so much cleaner and more elegant. I wanted this.

Hence, I created a series of new functions which maintained a minimal inner state where all the project files were able to call these functions and build projects functionally, rather than uses data structures. Take this example of how  looked before the new system: 

SDL_project = {
 name = "SDL2",
 kind = "SharedLib",
 language = "C++",
 dependencyTree = { },
 uuid = os.uuid(),
 sourcedir = "../src",
 -- as dependencies...?
 customLinks = { }
}

-- statically link on mac osx
if os.get() == "macosx" then
 SDL_project.kind = "StaticLib"
elseif os.get() == "windows" then
 table.insert(SDL_project.customLinks, "winmm")
 table.insert(SDL_project.customLinks, "imm32")
 table.insert(SDL_project.customLinks, "oleaut32")
 table.insert(SDL_project.customLinks, "version")
end

projects["SDL2"] = SDL_project

-- dependency functions must return the following:
-- <foundDep> <name> [includes] [libs] [inputs]
function directXDep()
 print("Checking DirectX dependencies...")
 local foundInc, incpath = find_dependency_dir_windows("DXSDK_DIR", "C:/Program Files;C:/Program Files (x86)", "DirectX", "Include")
 local foundLib, libpath = find_dependency_dir_windows("DXSDK_DIR", "C:/Program Files;C:/Program Files (x86)", "DirectX", "Lib/x86")
 if not foundInc or not foundLib then return false, "DirectX" end
 return true, "DirectX", { incpath }, { libpath }, { }
end

-- TODO: convert this to be functional (like premake), so the syntax isn't as
-- repetitive

-- format is { dependencyLambda }
-- if not in table, it will be excluded from the project
-- if dependency lambda is nil, it will always be included
local dep = SDL_project.dependencyTree;
-- setup dependency tree for SDL 2
dep["/"] = { nil }
dep["/atomic/"] = { nil }
dep["/audio/"] = { nil }
...
dep["/video/"] = { nil }
dep["/video/dummy/"] = { nil }
-- platform-specific implementations
if os.get() == "windows" then
 dep["/audio/directsound/"] = { nil }
 dep["/audio/winmm/"] = { nil }
...
 dep["/render/opengl/"] = { nil }
 -- added exclusion filter to thread/generic to avoid double linking warnings
 -- and incorrect linking
 dep["/thread/generic/"] = { nil, files = { "SDL_syscond.c", "SDL_sysmutex_c.h" } }
 dep["/thread/windows/"] = { nil }
 dep["/timer/windows/"] = { nil }
 dep["/video/windows/"] = { nil }
elseif os.get() == "macosx" then
 dep["/audio/coreaudio/"] = { nil }
 dep["/file/cocoa/"] = { nil }
...
 dep["/video/cocoa/"] = { nil }
 dep["/video/x11/"] = { nil }
else
 print("SDL2 via premake is not supported on platform: " .. os.get())
end

That was the old file, shortened. It's obviously disgusting and ugly. At least, it is compared to the newer version:

function directXDep()
 print("Checking DirectX dependencies...")
 local foundInc, incpath = find_dependency_dir_windows("DXSDK_DIR", "C:/Program Files;C:/Program Files (x86)", "DirectX", "Include")
 local foundLib, libpath = find_dependency_dir_windows("DXSDK_DIR", "C:/Program Files;C:/Program Files (x86)", "DirectX", "Lib/x86")
 if not foundInc or not foundLib then return false, "DirectX" end
 return true, "DirectX", { incpath }, { libpath }, { }
end

function winmmDep()
 print("Checking winmm dependencies...")
 local libpath = os.findlib("winmm")
 local foundLib = libpath ~= nil
 if not foundLib then return false, "winmm" end
 return true, "winmm", { }, { libpath }, { "winmm" }
end

SDL_project "SDL2"
 SDL_kind "SharedLib"
 SDL_language "C++"
 SDL_sourcedir "../src"
 -- dependency tree for SDL2
 SDL_dependency ""
  -- unnamed means it's not a dependency
  -- no OS means platform-independent

  -- this is a minimal setup that should
  -- essentially work on every target platform
  SDL_paths
  {
   "/",
   "/atomic/",
   "/audio/",
   "/audio/disk/",
   "/audio/dummy/",
   "/cpuinfo/",
   "/events/",
   "/file/",
   "/haptic/",
   "/joystick/",
   "/power/",
   "/render/",
   "/render/software/",
   "/stdlib/",
   "/thread/",
   "/timer/",
   "/video/",
   "/video/dummy/"
  }
 -- windows dependencies
 SDL_dependency "windows"
  SDL_os "windows"
  SDL_links { "imm32", "oleaut32", "version" }
  SDL_paths
  {
   "/core/windows/",
   "/haptic/windows/",
   "/joystick/windows/",
   "/libm/",
   "/loadso/windows/",
   "/power/windows/",
   "/render/opengl/",
   "/thread/windows/",
   "/timer/windows/",
   "/video/windows/"
  }
  SDL_files
  {
   -- these files have to be specified uniquely to avoid double
   -- and incorrect linking
   "/thread/generic/SDL_syscond.c",
   "/thread/generic/SDL_sysmutex_c.h"
  }
 -- winmm dependency
 SDL_dependency "winmm"
  SDL_os "windows"
  SDL_depfunc(winmmDep)
  SDL_paths { "/audio/winmm/" }
 -- directx dependency
 SDL_dependency "directx"
  SDL_os "windows"
  SDL_depfunc(directXDep)
  SDL_paths
  {
   "/audio/directsound/",
   "/audio/xaudio2/",
   "/render/direct3d/"
  }
 SDL_dependency "macosx"
  SDL_os "macosx"
  SDL_paths
  {
   "/audio/coreaudio/",
   "/file/cocoa/",
   "/haptic/darwin/",
   "/joystick/darwin/",
   "/loadso/dlopen/",
   "/power/macosx/",
   "/render/opengl/",
   "/thread/pthread/",
   "/timer/unix/",
   "/video/cocoa/",
   "/video/x11/"
  }

The beauty of this new code goes without explanation, but I will point some things out. I didn't have to trim it at all in my paste here, because it's hardly longer than the trimmed other version. The code length is shorter and there's less to type. I also increased the length of this version by separating the winmm dependencies from the general windows dependencies. The flexibility here is self-explanatory.

Complex SDL Configurations

I mentioned or hinted a while back at wanting more sophisticated means of building SDL. An example I gave was the ability to build SDL without any sort of native renderer, and having it still work. Ie, the windows solution file for SDL should not depend on OpenGL or DirectX at all. If someone doesn't have DirectX installed, they should still be able to build SDL. Granted, they will be missing a significant number of features, but that doesn't mean it shouldn't build, link, and run correctly.

However, simply omitting a few libraries from SDL is not enough to stop it from expecting those links. If I do not include /render/opengl, it will compile fine, but it won't link fine. Why? Because SDL is expecting the functions in /render/opengl to be there. How do we stop SDL from expecting something we don't want to be there?

Enter configure and autotools.

SDL's "assumption system" is based on a single configuration header file, which differs from operating system to operating system (on windows, it's SDL_config_windows.h). On a Unix environment, one can use the configure script to check for system dependencies and generate an appropriate config header file that allows SDL to build properly. This is very nice, but it was not taken into consideration for Windows or Mac OS X.

I've implemented a new function in the functionally-driven project definition system for setting up custom defines that can be pasted into a generated config file. For example, the following configuration directive is for setting up the defines needed to enable DirectX support in SDL:

  SDL_config
  {
   ["SDL_AUDIO_DRIVER_DSOUND"] = 1,
   ["SDL_AUDIO_DRIVER_XAUDIO2"] = 1,
   ["SDL_JOYSTICK_DINPUT"] = 1,
   ["SDL_HAPTIC_DINPUT"] = 1,
   ["SDL_VIDEO_RENDER_D3D"] = 1
  }

The only thing next to do is to decide whether to generate a config file or try to use compiler-level defines to simply set flags. Due to various defines in the SDL_config_windows.h file, I cannot just set compiler-level defines, unless I change that file to use #ifndef when setting some of the defines (it does it for many of the other defines, which is nice).

After that is decided, I'll be able to support a lot more flexibility when building SDL than was possible before on both Windows and Mac OS X. Also, if the build system is extended to other platforms, it will conveniently similar to the processing of configure and autotools for generating a makefile, if developers are interested in that.

Until next time,

Ben