Tuesday, May 26, 2009

SMASH Status

Last edited on 25.06.2009

This document is a more technical description of where SMASH stands, describing the internal flow and architecture. Please make sure you have read the Introduction first. If you are looking for an interesting read, I also recommend Thor Alexander's two books "
Massively Multiplayer Game Development", I use them very heavily for SMASH's development, knowing the books will make it easier to get a grasp on certain concepts. This document will change over time as progress is made, so today's read will be different from tomorrow's and then again, maybe it won't.

The Goal of SMASH
Once SMASH is finished you will be able to create a Massive Multiplayer Online Game (MMO), a messenger-, a dynamic webpage-, an IRC- or SMS-network or anything that requires sending messages across groups of users on different servers, while being completely independent of the platform, as long as Erlang compiles on the OS. SMASH can be upgraded in run-time without missing a beat. The application behavior is dictated by the channel controlling NPCs or Sim Proxies (see below).
This is why you will hear me call SMASH a "framework" here and there, because it's mostly a generic communication network on which you can build whatever functionality you require.

The SMASH Concept
At its very heart, SMASH is a chat network, which was inspired by Joe Armstrong's e
xample from "Programming Erlang: Software for a Concurrent World". Joe be at ease, I replaced all your good code with all new, lousy code.

But anyway, every event in SMASH is essentially a messa
ge sent from one participant to another. Messages are always tell commands, either to another player or a virtual player, also called NPC. The means of interaction with several other players and with the game world is via chat groups, which are controlled by NPCs, those NPCs are fundamentally the same object than a player proxy, but with a different plugin to control its behavior. To chat in channel "X" the player sends a tell to the NPC "X" which validates, if needed, the intended message and then relays it to all other subscribers of that channel "X". So, for simple chat channels those NPCs might be doing nothing else than relaying text or they might filter offensive messages and raising flags or mute a player for a while. NPCs will handle zone behaviors, by receiving requests from players, validating them and relaying resulting actions; other NPCs in that zone will be implemented as simple scripts or as other NPCs, that receive messages like any other player and act upon them. At one point in the far future, all NPCs might be virtual players, have a decent AI and really "live" in this simulated world.

Zone NPCs might want to communicate with a Physique server to validate movement action or with other zone NPCs for overlapping zones, or maybe you have a server where a simulation runs and have that simulation feed its data into SMASH; Leo, aut
hor of Apocalyx has done a great many demos, so maybe I will modify some of his demos to transmit their actions via TCP/IP and players would be able to watch one of his soccer simulation, or car races or robot fights. Time will tell.

How does a single SMASH node work

In practice a single commands in the Erlang shell starts the master node which triggers certain start-up functions and initializes the framework. Each new node starts up as a slave node, the services get started from the master node automatically.

A client connects to the SOX server (SOX) which returns a client socket for each new connection and notifies the chat server (CS) module of it. The CS starts a new chat client (CC) or client proxy, which is the representation of a player in the framework, so if someone disconnects his virtual representation persists for a programmer defined interval. So if someone disconnects during a fight, his toon will persist and most likely die, disconnecting won't save him, some online games omit this. The CC can have a plugin (action handler) linked to it that will validate incoming messages, so one moment the toon is under player control, the next it might be doing some scripted action, or a NPC lives in this game world driven by this kind of plugin support.

The-soon-to-be authorization module will automatically join the player to the channels he needs to be in and grant him the authorized commands with which he can operate in this virtual world. For each group a specialized NPC is started that controls the behavior of this channel (Sim Proxy). It might load a Logic Module for it, connect to a Physique server, load scripts to simulate objects in the zone, a dictionary to filter offensive words or even connect via TCP/IP to an application written in your favorite programming language. At some point some NPCs will need to start by themselves to simulate an on going economy, weather or connect to another simulation.

The command structure will be
{CHANNEL:REQ:PARAM1;PARAM2; ... PARAMn} from the client side, where the client can only send requests, the CC will validate if a user is authorized to that request command and if so forward it to the CNPC, which will be the unique instance to convert that request into some action and send a chat command back to each client in its channel. The CC will pick this up, modify its value on the serve side and send back the command to the client, which handle the commands with the following structure: {CHANNEL:WHO:{CMD1:PARAM1;PARAM2; ... PARAMn}{CMD2:PARAMS}{CMD3:PARAMS}} . This protocol is currently plain text for easy trouble shooting.

Underlying to the whole framework are some services like the Observer (OB) and Load Balancer (LB) that will restart broken services, auto start code on new nodes and decide where processes should run according to available resources. The lowest layer so far is the database, which will be migrated from Mnesia to KVS*, the Key Value Services. The envisioned structure is KVS for temporary NPC data or to cache behind the result of a function call to speed up certain things. Its data is also replicated among all nodes by KVS2. KVS2 on the other hand is a RAM resident, self replicating database. It saves data periodically and reorganizes by itself should the master node fail. In the future the OB and LB will follow that behavior to create a fully self reorganizing network. All relevant data will be thrown from local loop data to KVS/KVS2 to enable a swift restart of fallen services without any data loss. KVS* requires that SMASH follows the WORM principle (write once, read many) where only one writes a record, but any other can read it, several processes writing to the same record will mostly certainly lead to disaster and chaos. On the other hand KVS* enables SMASH to have a fast replicating database without write locks. KVS3 does not exist as of now and maybe never will. The 3rd database layer is always handled in a separate module, programs must never write directly to the master database. Currently this 3rd layer is mnesia, in the future it may be KVS3 or a separate database server cluster which will be accessed through a TCP/IP, bundling access to that database into one module makes it easy to change without breaking everything. This layer is meant for persistent data that requires 100% accuracy and has no special speed requirement, like billing data, realm data, account information etc. This layer will be fully implemented when everything else works.

As you can see, there are not all that many pieces to the puzzle, the intention is to have as little and as few but versatile parts as necessary. And from those right-sized buildings blocks we shall make something large, yet manageable (hopefully). Still missing here are AI, state machines, scripts, ... you name it, a lot of stuff.

How do several SMASH nodes work together

As stated above SMASH is meant to handle many servers as one unit, so on each node the same services run and communicate through the unique Sim Proxy to each local chat group object, which in return notifies each player or virtual player (maybe smart and not zone bound NPCs).

Erlang-wise these nodes need to be able to ping each other. This inter-node behavior already works. What's more, you can compile and update all of the code in run-time without missing a beat and given that Erlang code is platform independent, SMASH couldn't care less if a node runs on Windows, Mac, Linux, BSD or "Your-favorite-OS-here", a small tip here, linux sockets have proven to be roughly 3x faster than Windows sockets and beginning NT4 Workstation Windows sockets will only serve up to 10 connection concurrently (on desktop versions), while others will stay idle, I don't think this has changed, so you will need to use a server version to handle many clients in parallel or use some other platform for massive amounts of connections.

The vision of the finished SMASH framework

This picture shows clients connecting to a global TCP/IP server which reads global data and decides where the user was at last. So each SMASH cluster might be a zone, an instance, a realm, a site or a different game altogether. The clients would always see the same IP it connects to, hiding the full infrastructure behind it, although this is still talking about the far future, this might well be the culmination of the whole project, a sort of "SMASH Games Googolplex", let's call it SMAGG in short for future reference.

At this point I have no idea how easy it will be to write a game in SMASH, once the CC plugin support is due, I will see to this, SMASH will most certainly not be for script kiddies, but rather for seasoned developers with advanced skills, but I sure hope that the framework will lift the tons of weight for you to make a large scale project happen. Once finished I will write a Database editor that will hopefully have the power to create a full game, we'll see.

The current size of SMASH is under 1MB (source +compiled programs), so as you can see Erlang rocks when it comes to code size and I believe even once finished the compiled programs will be around 1 MB. The largest part will be most definitely the database.

Currently undecided is whether it's going to be open-sourced, if it will be worth while writing a book about or if I'll just scratch everything and plant a tree or two.

Contributions or how you could help

Well, while I have no real time table, given that this is just another hobby, you might be able to help, if you have a/some public domain ... :

1) city sized BSP level(s) I might use as a zoned demo level(s).
2) action figures for the demos, be it cowboys, spacetroopers, aliens, elves, etc.
3) any kind of 3D objects, animals, furniture, etc.

I am not really a 3D artist, so any free material might help, if not I'll just use Apocalyx demo material and when releasing a first playable demo ask the user to use all of Leo's .DAT files.