I have been working quite a bit in Python and Lua last year and I also tried to use those languages in our current game project at exDream (will be announced shortly, so I'm finally allowed to talk about it ^^). While both Python and Lua are written in C (there are obviously other ports like the great IronPython project for .NET, but I'm not going to talk about that in this post), it was not easy at all to make Python run on the Xbox 360 or PS3. I was able in early testing to get some parts running on the Xbox 360, but too much was missed and commented out and the PS3 port looked like way too much work. So I decided to just use Lua, which is probably a better choice for our game anyway since I only use it to do some AI and physics code with it along with some settings. I'm not going to benefit from the Python libraries either way, they are not needed for our game and they won't run on those consoles without serious effort anyway.
Ok, so I was writing some Lua code and used mostly LuaInterface to make the Lua code run from .NET and of course to work with all my .NET code and even XNA. The second step was to make this run in our new game engine, which is written in native code (C++) since the whole engine is based on the Unreal 3 Engine. For that I used the normal Lua 5.1.4 release, which compiles just fine and works great. Since Lua is written completely in ANSI C I was able to quickly test some code on the Xbox 360 and PS3 too. This is important because our game will be a multi-platform title (PC, Xbox 360 and PS3), which is thanks to the Unreal 3 Engine a much simpler task than writing our own engine for all those platforms (it has obviously it own disadvantages, not only do we have to use native code, but we also have to work with a lot of code in the engine that we don't really need for our game).
So far so good. While it sounds like a simple task to make Lua run on all platforms, there is a lot of pain involved to make it run perfectly and behave the same way on all platforms. Since the Unreal 3 Engine is written in C++ and uses its own scripting Language UnrealScript, it does not help out a lot when you want to integrate your own code or libraries in there, so this was half of the work (just see my previous post about compile times and you see why this is a much lengthier process than writing some .NET code). Secondly there are many special rules on both the Xbox 360 platform and the PlayStation 3, some ANSI C methods are just missing or just don't do anything because of the platform design. For example on the Xbox 360 you can't use relative paths, you always have to specify a full path when opening a file, which is something that the Lua engine does not do out of the box. After figuring out where to store the Lua scripts on the consoles and how to load them, everything else just falls into place and works finally as expected.
To make things simpler for other people trying to run Lua code on the Xbox 360 or PS3 I wrote the following little solution:
As you can see on the right the solution consists of 4 projects:
- LuaSupport, which is based on the current Lua 5.1.4 release from 22 Aug 2008 with my own additions to support the XBOX and PS3 platforms (uses XBOX and PS3 defines).
- TestLuaPC is a simple test project for the PC to make sure everything still runs as expected on the PC. Please note that on the PC Lua assumes that all files can be found in the current directory. I just change to the Lua script directory and then execute the Lua files there normally.
- TestLuaPS3 will probably only compile if you have the PS3 SDK installed, just unload this project if you don't have it installed yet. Also please test other SDK samples first to make sure everything is correctly setup (it uses a lot of extra include paths). In case you are not able to open the files, check for yourself if the /app_home/Lua/ directory contains the required files after building the project (use the ProDG Target Manager for that). Debugging can also be complicated, I guess it does not work on Windows 7 at all, but I was able to debug and run the app on a normal Windows Vista PC. In any case you can also just run the .elf file from the ProDG Target Manager and then see the output in the console.
- TestLuaXbox360 is the project to test Lua on the Xbox 360. You will also need the XDK (Xbox Development Kit SDK) installed and have a Xbox 360 DevKit in your network for testing. A TestKit also works, but you won't be able to debug, you can just run the app and check the console output yourself.
Some of the Lua functionality is not available on the Xbox 360 or PS3 like lua_popen or lua_pclose (among others), and there is also some code that will not do anything like os_execute, os_getenv and some other os methods because they won't make sense on a console anyway or could not be implemented. I did not use these methods anyway, but if you need those and want to write your own code for them, go ahead.
Lets focus on the 2 most important code changes. First of all the Lua default directories had to be adjusted in luaconf.h:83 to support Xbox 360, PS3, PC and Linux platforms:
#if defined(XBOX)
/* On the Xbox we just use the D:\ drive for everything, all lua files should be
located there */
#define LUA_LDIR "D:\\Lua\\"
#define LUA_CDIR "D:\\"
#define LUA_PATH_DEFAULT \
LUA_LDIR"?.lua;" LUA_LDIR"?\\init.lua;" \
LUA_CDIR"?.lua;" LUA_CDIR"?\\init.lua"
#define LUA_CPATH_DEFAULT \
".\\?.dll;" LUA_CDIR"?.dll;" LUA_CDIR"loadall.dll"
#elif defined(PS3)
/* On the PS3 we just use the /app_home/Lua/ sub directory for everything,
all Lua files should be located there */
#define LUA_LDIR "/app_home/Lua/"
#define LUA_CDIR "/app_home/"
#define LUA_PATH_DEFAULT \
LUA_LDIR"?.lua;" LUA_LDIR"?\\init.lua;" \
LUA_CDIR"?.lua;" LUA_CDIR"?\\init.lua"
#define LUA_CPATH_DEFAULT \
".\\?.dll;" LUA_CDIR"?.dll;" LUA_CDIR"loadall.dll"
#elif defined(_WIN32)
/*
** In Windows, any exclamation mark ('!') in the path is replaced by the
** path of the directory of the executable file of the current process.
** Note: We use the ..\Lua\ directory by default, but that is also the current
** directory, we change it in AIEngineInterface before running scripts!
*/
#define LUA_LDIR "!\\..\\Lua\\"
#define LUA_CDIR "!\\"
#define LUA_PATH_DEFAULT \
".\\?.lua;" LUA_CDIR"?.lua;" LUA_CDIR"?\\init.lua;" \
LUA_LDIR"?.lua;" LUA_LDIR"?\\init.lua"
#define LUA_CPATH_DEFAULT \
".\\?.dll;" LUA_CDIR"?.dll;" LUA_CDIR"loadall.dll"
#else
/* Default linux/mac directories */
#define LUA_ROOT "/usr/local/"
#define LUA_LDIR LUA_ROOT "share/lua/5.1/"
#define LUA_CDIR LUA_ROOT "lib/lua/5.1/"
#define LUA_PATH_DEFAULT \
"./?.lua;" LUA_LDIR"?.lua;" LUA_LDIR"?/init.lua;" \
LUA_CDIR"?.lua;" LUA_CDIR"?/init.lua"
#define LUA_CPATH_DEFAULT \
"./?.so;" LUA_CDIR"?.so;" LUA_CDIR"loadall.so"
#endif
And the most important code change is located in luaL_loadfile method, which is called by the lua_dofile macro. Please note that I did not change the code formating and tried to leave as much code as is. I have my own coding conventions, but it does not make sense to apply them when only changing <10% of the code of an existing library.
LUALIB_API int luaL_loadfile (lua_State *L, const char *filename) {
LoadF lf;
int status, readstatus;
int c;
char fullFilename[128];
int fnameindex = lua_gettop(L) + 1; /* index of filename on the stack */
lf.extraline = 0;
if (filename == NULL) {
lua_pushliteral(L, "=stdin");
lf.f = stdin;
}
else {
// Note: Always open as the requested file, Lua should not care about
// our crazy directory remapping.
lua_pushfstring(L, "@%s", filename);
#if (PS3)
// On the PS3, check always the /app_home/Lua/ directory!
// This works both for dev-testing via remote HDD and on the game disc.
// These work probably too just for developing, but app_home is fine!
// /host_root/Lua/ or /dev_bdvd/PS3_GAME/USRDIR/Lua/
CheckForValidFilenameWithPath(fullFilename, filename, "/app_home/Lua/");
#elif XBOX
// For the Xbox we have to make sure always to load files from D:\ because
// fopen ALWAYS expects a full paths, there are no current directories on the
// Xbox 360 and therefore no relative paths! Check always "D:\Lua\<file>"
CheckForValidFilenameWithPath(fullFilename, filename, "D:\\Lua\\");
#else
// On the PC we just use the default search logic (see luaconf.h) and we
// don't care about directories since we will already be in the correct one!
// In earlier versions we had a lot of extra checks here.
strcpy(fullFilename, filename);
#endif
// Rest of this code is untouched, we just use fullFilename now!
lf.f = fopen(fullFilename, "r");
if (lf.f == NULL)
return errfile(L, "open", fnameindex);
}
c = getc(lf.f);
if (c == '#') { /* Unix exec. file? */
lf.extraline = 1;
while ((c = getc(lf.f)) != EOF && c != '\n') ; /* skip first line */
if (c == '\n') c = getc(lf.f);
}
if (c == LUA_SIGNATURE[0] && fullFilename) { /* binary file? */
lf.f = freopen(fullFilename, "rb", lf.f); /* reopen in binary mode */
if (lf.f == NULL) return errfile(L, "reopen", fnameindex);
/* skip eventual `#!...' */
while ((c = getc(lf.f)) != EOF && c != LUA_SIGNATURE[0]) ;
lf.extraline = 0;
}
ungetc(c, lf.f);
status = lua_load(L, getF, &lf, lua_tostring(L, -1));
readstatus = ferror(lf.f);
if (filename) fclose(lf.f); /* close file (even in case of errors) */
if (readstatus) {
lua_settop(L, fnameindex); /* ignore results from `lua_load' */
return errfile(L, "read", fnameindex);
}
lua_remove(L, fnameindex);
return status;
}
The code in luaL_loadfile will call the new method CheckForValidFilenameWithPath, which helps us to build a full path filename for the Xbox 360 and PS3 platforms. It will just add D:\Lua\ or /app_home/Lua/ in front of the filename if it does not exist yet. Here we can see another pain point of C code, it takes so freaking long to do very simple tasks (in .NET this would be a 2-liner).
static void CheckForValidFilenameWithPath(char* fullFilename,
const char* filename, const char* platformPath)
{
int i = 0;
int pathLen = (int)strlen(platformPath);
// Already have a filename with the correct path?
if ((int)strlen(filename) > pathLen)
for (i = 0; i < pathLen; i++)
if (filename[i] != platformPath[i])
break;
// Found path?
if (i == pathLen)
{
// Then just use the existing file, happens when loading libraries from Lua
strcpy(fullFilename, filename);
}
else
{
// Copy path
for (i = 0; i < pathLen; i++)
fullFilename[i] = platformPath[i];
// Add the relative filename
for (i = 0; i < (int)strlen(filename); i++)
fullFilename[i+pathLen] = filename[i];
// And finally close the string
fullFilename[(int)strlen(filename)+pathLen] = 0;
} // else
} // CheckForValidFilenameWithPath(fullFilename, filename, platformPath)
And that are the most important code changes to make everything run on the Xbox 360, PS3 and PC. Here is the main method for all the TestLua projects (always pretty much the same, only the Xbox 360 uses a slightly different way to print to the console):
int _tmain(int argc, _TCHAR* argv[])
{
printf("Initialize Lua\n");
Lua = lua_open();
luaL_openlibs(Lua);
lua_register(Lua, "print", lua_print);
// Simple print test from Lua:
luaL_dostring(Lua, "print(\"Hi from Lua\")");
// Now load a Lua file, which also loads another Lua file (LuaUnit.lua)
if (luaL_dofile(Lua, "SomeClass.lua") != 0)
{
printf("Error: Unable to execute Lua file %s: %s\n", "SomeClass.lua",
lua_tostring(Lua, 1));
} // if (luaL_dofile)
// Stop the Lua engine and cleanup
printf("Closing Lua\n");
lua_close(Lua);
Lua = NULL;
// Keep command window for debug mode (ctrl+f5 does not have this issue)
char input[100];
gets_s(input);
return 0;
} // _tmain
Summary:
- Using Lua on other platforms is easier than Python or other dynamic languages I have checked because Lua is written completely in ANSI
- Making code run on multiple platforms is still hard in C++ because you not only end up with ugly #define code, but you also have to test a lot
- Building native applications is freaking time-intensive, it took me almost a full day to write this app and test it on every platform from serval PCs. In .NET it would take me less than an hour, but then again there is no .NET on the PS3 and our current game engine is using native code anyway.
- On the Xbox 360 you always have to use full paths like D:\Lua\SomeClass.lua, relative paths won't work. The SDK says you should use Game:\, but since our game engine uses D:\ too, I left it that way in this sample too.
- The PS3 is even more complicated with lots more possible mounted paths, but it seems /app_home did always work in both my samples and our game, so I'm sticking with it :)
If you are interested in Lua from .NET, check out the LuaInterface project. Someone also started NUA (Lua on the DLR) 2 years ago, but it seems this project (and many other attempts) have died a long time ago. There was also a port by the Lua guys (especially the great Fabio Mascarenhas, who has also started the LuaInterface project and is still involved) called LuaCLR, but it is still in alpha stage and pretty useless, LuaInterface on the other hand is a fully working library! The difference here is that LuaCLR emits CLR code like any other .NET language, while LuaInterface still relys on the Lua runtime. LuaInterface uses only 2 very simple C++ files, the rest is in C#, but because of that you currently cannot run it with XNA on the Xbox 360. IronPython, while fully written in C#, does not work on the Xbox 360 either because reflection is not supported by the Xbox 360 .NET framework.
And finally here is a link to Lua running on the iPhone: Lua v5.1.3 for iPhone/iPod, kinda cool, but since we can run C# code on the iPhone with help of the great Unity3D engine, I'm happy with that at the moment. The main problem for other languages on the iPhone is the missing libraries anyway (can't use Cocoa or any own UI) ..
Just in case you have not read about it yet: John Carmack has ported the good old Wolfenstein 3D game from 1992 to the iPhone as well.