Need for speed

When doing the first message bombardment on the server and the news are mixed ones. On the one hand the server does not crash, which is nice, but after a while it starts to behave irrationally, like when logging out and logging back in, it does not seem to delete users from the DB anymore, so a kind of measurement is needed. Writing some quick benchmarks it turns out that the Chat Server (CS) can do 600'000 messages per second (mps) and the DB lookup in the chat group can only do about 6'800 mps, with 16 clients connected we get up to 2'600 mps, but I suppose that the constant bombardment becomes a problem at some point, because Erlang starts piling up messages and the missing client deletion from the DB is most likely the same cause, so a first conclusion is that Mnesia can't keep up with all those lookups on the DB and this causes messages to pile up like crazy. Fortunately this was likely to happen and now we have a legal excuse to use the KVS (Key Value Server), which has been completely dormant until now. The KVS will need to grab this kind of data from Mnesia and put it in a local table. The big difference now will be this, the KVS will only load the data from Mnesia every "x" time intervals, like maybe every second or every 5, meaning that we will lose precision in order to gain speed. We will now have to segregate our operations, those that require ultimate precision like close by moveable objects, banks, auction houses, trades and loot operations and others like chat, or even general environment operations that do not require the most precision. And we need to start with the Chat Group structure. This should be really easy to do, we will see if things speed up afterwards. A general throughput of 500'000 mps looks like our boy, but 7'000 mps does not.

And surprise, surprise, the Playstation 3 is not nearly as powerful as hoped for as a server, it's depending on the test 4-10 slower than a Dual core Windows Laptop, averaging so far like 6x slower, how very disappointing. In the CS test it scores at 95'000 mps, the DB lookup only gets 4'400 mps and it takes 24 sec. to create 10'000 CC clients, while the Laptop does that job in 7 sec., OUCH !!!!!

Long live the benchmarks.

Sunweaver

First benchmarks

We have conducted the first couple of test series on a single-core notebook (sc), a dual-core notebook (dc) and the PS3, where creating 10'000 users takes 1:11 min., 0:22 min and 0:42 min respectiviely. We then proceeded to send chat messages between the 3 systems and the results are similar in ratio. What is very apparent is that the PS3 displays the text relatively slowly, so in my opinion given that the CEL processor should be very superior to the dual-core it seems that the display is what makes it slow. We will conduct a modified test series, without the display part to check on performance. There is also the aspect of the hard disk performance, but the both series, user creation and chat showed more or less the same difference and user creation requires hard disk access, while chatting does not, so the difference of the 5400 rpm on the PS3 and the 7200 rpm on the dc do not seem to make that much of a difference.

Last not least, it may be that Erlang is not really optimized for CEL processors and maybe it does not distribute the processes correctly, but our personal appreciation is that the display is the bottleneck.

We will see what happens next.

Sunweaver

PS3 + YDL + Erlang = ready for prime time

Ok, it's been a long Sunday, but I did manage to install Yellow Dog Linux on my Playstation 3 successfully. It took much longer to backup the PS3 than it did to configure YDL. After that I installed VNC Server to remote control the machine and let me tell you, that the Gnome Desktop is a beauty. Installing Erlang was as simple as downloading and compiling it, so no new really on this end, either. In other words, the server platform is ready for deployment, I will now go ahead and run the SMASH framework on it to compare speed ratios among my test systems, let's see what happens next. This is for me a major milestone, given that THIS PS3 will be the mother of all SMASH development nodes. I will have some time next week as well to push the conection between the socket server (Mr. SOX) and the SMASH framework, so I definitely hope to have some empty game world that people can actually connect to by next week. If that holds true, we might do a very first game world logon, maybe, maybe, maybe.

Oh and by the way, some may ask, WHY Linux and PS3 at all ??? Very simple, the thought process behind that is, that I want to be able to provide a server frame work taht is inexpensive and powerful, because regular servers are definitely more $$$ heavy and not everybody will be able to afford buying Windows Server licenses, so nothing beats Linux moneywise and a U$500 PS3 server should make a nice duo. Now, that much said, many people are scared of Linux, because it's not exactly intuitive, well, given that we only need a DVD install and later on a simple download + install of Erlang, that should be fairly simple to do for everyone and ALL the others tasks can be done thru native Erlang, no more touching of the PS3 is required, I am planning on adminstrating everything from a more comfortable platform, without losing the power of the solution.

Stay tuned.

Sunweaver

What's next ??

Alright, it's 2008 and I finally have some time to come back to the SMASH project. One of my goals was to buy a PS3 and install Linux on it, so at last I got me one and I will install Yellow Dog on it this weekend. Why a PS3 ?? The processor of the PS3 has internally 6 cores that are ready for use and it is said to be a very powerful server, so I will set up Erlang on it and see just how fast it goes. Erlang recognises multiple cores, so this should prove to be a powerful combination, a dual-core versus single-core scores about 40% speed increase, so 6 internal cores should make quite a difference. I will post how it went, both installation and benchmark.

I am also working on the TCP/IP server and I should soon have a first first alpha server up and running where people can connect to (probably a pretty empty world).

See you soon.

Sunweaver