More Configurations, MinGW, and Midterm for GSoC

Howdy,

Halfway done with Google Summer of Code. I have a few quick updates to discuss and then I have a request for the SDL community.

Complex Configurations (pt. 2)

I discussed in my last post how I added support for complex configurations by allowing SDL projects to set preprocessor definitions that would be embedded with a configuration header file that drives the entire SDL build process. This is the same way taken by ./configure and related Linux systems. The system has been implemented at this point and has allowed me to do some interesting stuff.

First of all, the modularity of SDL is much more customizable now than it was previously. I can disable DirectX or OpenGL support with ease. If the meta-build system cannot find one or both of these dependencies, it will appropriately configure SDL to neither compile its related modules nor link to the resulting modules. It becomes rather convenient. I theorized in the beginning of my project that SDL should be able to be built and used on Windows without DirectX support. Recently, I have successfully built and ran all the tests without DirectX support. It works surprisingly well. There are a feel hiccups in the test suites, but that's expected given this was mostly untested behavior prior to this meta-build project.

I hope to expand the configurations systems to new depths by providing command-line arguments for easily enabling and disabling whole modules of SDL, without having to have any previous knowledge about its configuration systems or the meta-build system itself. As this project evolves to more a more modular design, this customization will prove to be very convenient and beneficial. SDL can literally be built dozens of different ways on some systems and I would really like to try and capture that with as much ease as possible. The only thing the end user would need to be aware of is which dependencies SDL has, which can be easily documented.

Full Test Suite

I forgot to mention this in my last post, but I did add support for building and linking the entire SDL test suite to every platform supported. Originally, the Visual Studio portion of SDL only had 10 tests supported. The Xcode version had a few more than that. It turns out there are 42 tests part of the SDL test suite. I was a bit shocked when I noticed I wasn't targeting most of them, so I worked to add support for the whole lot. I've written a lot of convenient build scripts (as I will discuss later) for making building and running these tests as painless as possible.

Mac OS X: Multiple Architectures

I am beginning to look into multiple architecture support, starting with Mac OS X. Currently, the Xcode projects should support building to architectures i386 and x86_64. I am planning on implementing Universal binaries for Mac OS X and any platform that has GCC, so including Windows through MinGW and Cygwin, as well as any Linux platforms. This is continuing the current support for "fat" binaries that SDL offers.

MinGW Support

I decided to be a bit adventurous this week. I implemented MinGW support using GNU makefiles. I hope to expand this project to full make support, but I needed to start with something that would not involve a lot of changes to the current project. MinGW is targeted for building native Windows applications using GCC and the GNU toolchain. Essentially, that implies the Windows setup to project generation is not much different between Visual Studio and MinGW. In some parts this is true, but in others...not so much.

The major differences are in the path separators and the different compilers (obviously). I had to create a MinGW mode (using the --mingw command line option) to signify using POSIX paths, etc. Beyond that, I ran into a lot of trouble over properly linking to the main function of the test applications. I had to use the linker argument -lmingw32 to GCC, though premake was giving me a lot of grief by making it only possible to put it in the wrong place. At the end of the day, I had to modify premake and change the order it generates its linker arguments in the resulting makefiles.

The only other problem with the MinGW support is GCC cannot compile the DirectX and Windows SDK header files correctly. There are MinGW alternatives to DirectX, but these are not the easiest things to work with. I've encountered some problems working with them and I know a bunch of other people have, too. As a result, I am not going to support DirectX with the MinGW generation option. I do plan on adding an override command line argument to the meta-build system for generating MinGW projects that still link to DirectX regardless of it being MinGW, leaving it up to the fate of the developer to try and get the thing to link properly.

Google Summer of Code Midterm!

I have to say I'm a bit surprised that it's already the halfway point. Time does fly when you're really focused on a project. Nevertheless, I am hoping the SDL community is able to help me at this point by pulling my repository and tinkering with it. Please, if you have time, try to give the meta-build system a whirl. The currently supported platforms are Windows using Visual Studio and MinGW and Mac OS X using Xcode. Here are a few things to note:

Obviously the system is very experimental. That's why it needs to be tested. If something goes wrong, feel free to email me, comment on this blog, or post it on the mailing list. I will see it and get back to you very soon. I appreciate any problems or feedback you have.

If you want to just give a quick contribution to the project, simply build the SDL project and test suites for whichever choice platform and IDE. If you run the tests, it can give a quick feedback about whether SDL seems to run mostly normally on your system using the build configurations setup by the meta-build system. I have created several build scripts for each platform for handling various types of builds, as well as testing those builds using the extensive SDL test suite.

If you want to give a more elaborate contribution, I would really appreciate you testing binary compatibility or interoperability by either building or linking any SDL projects you have or are using against the libraries produced by the meta-build projects. This would give excellent feedback as to whether I am correctly generating the projects and that they are behaving as they should so far.

I really appreciate any and all help so far.

Thanks,

Ben

Scrubbing Project Syntax & Complex Configurations

Greetings,

After a six day outing to South Dakota, I'm happy to return and get right back to work on SDL's meta-build system!

Cleaning up Project Syntax

Before, the project syntax was rather icky. It involved a lot of table manipulation, assignment, listing, you name it. I just didn't like it. I didn't plan on keeping it from the beginning. In fact, I wanted a syntax closer to premake's.

With premake, you just do a bunch of function calls in a specific order. However, it doesn't feel like you're making function calls. That's because in Lua, you don't need surrounding parentheses when making a function call if you're passing it one parameter and that parameter is either a table or a string. Conveniently, everything in premake takes either a string or a table. So, in all their examples, they demonstrate passing string literals and inline tables. This gives almost a functional programming feel (which is obviously the end goal on Lua's part). It's just so much cleaner and more elegant. I wanted this.

Hence, I created a series of new functions which maintained a minimal inner state where all the project files were able to call these functions and build projects functionally, rather than uses data structures. Take this example of how  looked before the new system: 

SDL_project = {
 name = "SDL2",
 kind = "SharedLib",
 language = "C++",
 dependencyTree = { },
 uuid = os.uuid(),
 sourcedir = "../src",
 -- as dependencies...?
 customLinks = { }
}

-- statically link on mac osx
if os.get() == "macosx" then
 SDL_project.kind = "StaticLib"
elseif os.get() == "windows" then
 table.insert(SDL_project.customLinks, "winmm")
 table.insert(SDL_project.customLinks, "imm32")
 table.insert(SDL_project.customLinks, "oleaut32")
 table.insert(SDL_project.customLinks, "version")
end

projects["SDL2"] = SDL_project

-- dependency functions must return the following:
-- <foundDep> <name> [includes] [libs] [inputs]
function directXDep()
 print("Checking DirectX dependencies...")
 local foundInc, incpath = find_dependency_dir_windows("DXSDK_DIR", "C:/Program Files;C:/Program Files (x86)", "DirectX", "Include")
 local foundLib, libpath = find_dependency_dir_windows("DXSDK_DIR", "C:/Program Files;C:/Program Files (x86)", "DirectX", "Lib/x86")
 if not foundInc or not foundLib then return false, "DirectX" end
 return true, "DirectX", { incpath }, { libpath }, { }
end

-- TODO: convert this to be functional (like premake), so the syntax isn't as
-- repetitive

-- format is { dependencyLambda }
-- if not in table, it will be excluded from the project
-- if dependency lambda is nil, it will always be included
local dep = SDL_project.dependencyTree;
-- setup dependency tree for SDL 2
dep["/"] = { nil }
dep["/atomic/"] = { nil }
dep["/audio/"] = { nil }
...
dep["/video/"] = { nil }
dep["/video/dummy/"] = { nil }
-- platform-specific implementations
if os.get() == "windows" then
 dep["/audio/directsound/"] = { nil }
 dep["/audio/winmm/"] = { nil }
...
 dep["/render/opengl/"] = { nil }
 -- added exclusion filter to thread/generic to avoid double linking warnings
 -- and incorrect linking
 dep["/thread/generic/"] = { nil, files = { "SDL_syscond.c", "SDL_sysmutex_c.h" } }
 dep["/thread/windows/"] = { nil }
 dep["/timer/windows/"] = { nil }
 dep["/video/windows/"] = { nil }
elseif os.get() == "macosx" then
 dep["/audio/coreaudio/"] = { nil }
 dep["/file/cocoa/"] = { nil }
...
 dep["/video/cocoa/"] = { nil }
 dep["/video/x11/"] = { nil }
else
 print("SDL2 via premake is not supported on platform: " .. os.get())
end

That was the old file, shortened. It's obviously disgusting and ugly. At least, it is compared to the newer version:

function directXDep()
 print("Checking DirectX dependencies...")
 local foundInc, incpath = find_dependency_dir_windows("DXSDK_DIR", "C:/Program Files;C:/Program Files (x86)", "DirectX", "Include")
 local foundLib, libpath = find_dependency_dir_windows("DXSDK_DIR", "C:/Program Files;C:/Program Files (x86)", "DirectX", "Lib/x86")
 if not foundInc or not foundLib then return false, "DirectX" end
 return true, "DirectX", { incpath }, { libpath }, { }
end

function winmmDep()
 print("Checking winmm dependencies...")
 local libpath = os.findlib("winmm")
 local foundLib = libpath ~= nil
 if not foundLib then return false, "winmm" end
 return true, "winmm", { }, { libpath }, { "winmm" }
end

SDL_project "SDL2"
 SDL_kind "SharedLib"
 SDL_language "C++"
 SDL_sourcedir "../src"
 -- dependency tree for SDL2
 SDL_dependency ""
  -- unnamed means it's not a dependency
  -- no OS means platform-independent

  -- this is a minimal setup that should
  -- essentially work on every target platform
  SDL_paths
  {
   "/",
   "/atomic/",
   "/audio/",
   "/audio/disk/",
   "/audio/dummy/",
   "/cpuinfo/",
   "/events/",
   "/file/",
   "/haptic/",
   "/joystick/",
   "/power/",
   "/render/",
   "/render/software/",
   "/stdlib/",
   "/thread/",
   "/timer/",
   "/video/",
   "/video/dummy/"
  }
 -- windows dependencies
 SDL_dependency "windows"
  SDL_os "windows"
  SDL_links { "imm32", "oleaut32", "version" }
  SDL_paths
  {
   "/core/windows/",
   "/haptic/windows/",
   "/joystick/windows/",
   "/libm/",
   "/loadso/windows/",
   "/power/windows/",
   "/render/opengl/",
   "/thread/windows/",
   "/timer/windows/",
   "/video/windows/"
  }
  SDL_files
  {
   -- these files have to be specified uniquely to avoid double
   -- and incorrect linking
   "/thread/generic/SDL_syscond.c",
   "/thread/generic/SDL_sysmutex_c.h"
  }
 -- winmm dependency
 SDL_dependency "winmm"
  SDL_os "windows"
  SDL_depfunc(winmmDep)
  SDL_paths { "/audio/winmm/" }
 -- directx dependency
 SDL_dependency "directx"
  SDL_os "windows"
  SDL_depfunc(directXDep)
  SDL_paths
  {
   "/audio/directsound/",
   "/audio/xaudio2/",
   "/render/direct3d/"
  }
 SDL_dependency "macosx"
  SDL_os "macosx"
  SDL_paths
  {
   "/audio/coreaudio/",
   "/file/cocoa/",
   "/haptic/darwin/",
   "/joystick/darwin/",
   "/loadso/dlopen/",
   "/power/macosx/",
   "/render/opengl/",
   "/thread/pthread/",
   "/timer/unix/",
   "/video/cocoa/",
   "/video/x11/"
  }

The beauty of this new code goes without explanation, but I will point some things out. I didn't have to trim it at all in my paste here, because it's hardly longer than the trimmed other version. The code length is shorter and there's less to type. I also increased the length of this version by separating the winmm dependencies from the general windows dependencies. The flexibility here is self-explanatory.

Complex SDL Configurations

I mentioned or hinted a while back at wanting more sophisticated means of building SDL. An example I gave was the ability to build SDL without any sort of native renderer, and having it still work. Ie, the windows solution file for SDL should not depend on OpenGL or DirectX at all. If someone doesn't have DirectX installed, they should still be able to build SDL. Granted, they will be missing a significant number of features, but that doesn't mean it shouldn't build, link, and run correctly.

However, simply omitting a few libraries from SDL is not enough to stop it from expecting those links. If I do not include /render/opengl, it will compile fine, but it won't link fine. Why? Because SDL is expecting the functions in /render/opengl to be there. How do we stop SDL from expecting something we don't want to be there?

Enter configure and autotools.

SDL's "assumption system" is based on a single configuration header file, which differs from operating system to operating system (on windows, it's SDL_config_windows.h). On a Unix environment, one can use the configure script to check for system dependencies and generate an appropriate config header file that allows SDL to build properly. This is very nice, but it was not taken into consideration for Windows or Mac OS X.

I've implemented a new function in the functionally-driven project definition system for setting up custom defines that can be pasted into a generated config file. For example, the following configuration directive is for setting up the defines needed to enable DirectX support in SDL:

  SDL_config
  {
   ["SDL_AUDIO_DRIVER_DSOUND"] = 1,
   ["SDL_AUDIO_DRIVER_XAUDIO2"] = 1,
   ["SDL_JOYSTICK_DINPUT"] = 1,
   ["SDL_HAPTIC_DINPUT"] = 1,
   ["SDL_VIDEO_RENDER_D3D"] = 1
  }

The only thing next to do is to decide whether to generate a config file or try to use compiler-level defines to simply set flags. Due to various defines in the SDL_config_windows.h file, I cannot just set compiler-level defines, unless I change that file to use #ifndef when setting some of the defines (it does it for many of the other defines, which is nice).

After that is decided, I'll be able to support a lot more flexibility when building SDL than was possible before on both Windows and Mac OS X. Also, if the build system is extended to other platforms, it will conveniently similar to the processing of configure and autotools for generating a makefile, if developers are interested in that.

Until next time,

Ben