The sun rises in ProcWorld, again

Last week I posted some early screenshots of the nigh-day cycle. There was a lot going wrong in them and you guys were very helpful in pointing out solutions.

I did another iteration on this, while not everything is as it should, I think there was some improvement. This time I have captured a video. The transitions are better appreciated like this. Again let me know what you think.

The main issues with the previous iteration were the brightness of the sky (or lack of it), and how the distant features failed to blend with the sky. This time I made sure there was enough atmosphere so more light was trapped between the horizon and the eye. The distance to the sky is also consistent with the terrain dimensions, now the colors in terrain and sky match better.

A few comments in the previous post suggested a different method called "Precomputed Atmospheric Scattering". It certainly produces better results than the method I am using here, which is the one from O'Neil in GPU Gems 2.

I had a quick look at the method and saw that in its vanilla form it could run slower than what I have now. While the method uses precomputed tables as textures to accelerate rendering, all the work is done in the fragment shader. That means every pixel on screen now had to perform two or three additional texture fetches.

The method from O'Neil does all the heavy lifting in the vertex shader. Consider this scene:

The sky, even if it appears softly shaded, has only a few vertices:

I think in this case it makes a big difference.

The precomputed method could also run in the vertex shader, but then it would take some time to port the tables, which now are in pixel formats that cannot be read by the vertex shader.

Of course there is a chance I am reading this wrong. If you have worked in this area before and see what I am missing please let me know.

The sun rises in ProcWorld

So I finally added some proper light scattering to the sky atmosphere in the realtime demo.

I am using the classic approach devised by O'Neil, which produces great results but it is also very sensitive to any change in the input parameters. A lot of tweaking is still required.

Here is a series of shots, keep in mind this is a work in progress but any early comments surely will help.

 

Network Update

Here is a new video I recently captured. It shows the networking and storage components in action.



When it comes to networking this is the smallest test possible, you cannot really go below two connected clients. I have tested this same server code with nearly a hundred clients performing queries and changes at rates many times higher than what humans would do. Network tests are good at showing why some stuff does not work. But when the results are good it does not really mean anything. The real network is so complex you cannot replace it by any model. In this case results are as good as any network test can be at this stage. There is very little overhead from the thread and connection management, which is what I was looking for.

While this is good news and by all means necessary, the real bottleneck comes from how any application using this engine chooses to store and process information. So again what you are seeing here is just a brick. You could create many different houses with it.

You could do it like Minecraft servers do, have everything including procedural generation run in the server. You could do like this particular demo does, where user-created content is stored in a server and everything else remains client-side. And you could have solutions in-between, for instance have some custom server-side generation which is merged later with the rest of the client-side generation.

This is a fascinating subject to me, I will be covering some of these approaches in the future.

Storage Matters

Imagine you were creating a massive persistent world where everyone would be able to change anything at will. It is a simple, powerful idea that eventually has occurred to everyone ever exposed to a game. Why there aren't many of these worlds out there? Well, this very simple idea is quite difficult and expensive to execute. Not only you need to store the information, you have to be able to write it and read it in a timely fashion.

Then how about your own personal world, something you can run in your PC and invite some friends to play over. How much of your PC's performance are you willing to sacrifice, how many people could you actually invite before you would see the quality of your gameplay begin to suffer?

I began wondering whether all the above could be manifestations of the same problem. What if you could have a storage solution that is lightweight so enthusiasts could run at home, and if you pieced enough of them together you could scale it so it would run massive worlds the size of planet Earth?

As it turns out it was possible. I have now a shinny new database system that does exactly that. The main trick is it aligns with the same other concepts of the voxel world. So this is mainly a voxel database. It won't do any SQL queries, XPath evaluation or any other form of traditional DB interaction. It just stores and retrieves voxel data very fast.

How fast? Over a 10 minute period, a machine with six-year-old Intel processor (T2500 at 2GHz) and an equally crappy HD was able to serve 10 Gigabytes worth of individual queries while another 10 Gigabytes worth of queries were being written. Each query ranged from 500 bytes to 100KBytes worth of data.

That would translate into a lot of friends sharing your server. To give you a better idea, a volume of 40x40x40 worth of player voxels compresses to 2K as an average. Here is how you would compute how much space 10 GB of voxel data would be:

1 chunk = 40x40x40 voxels = 12x12x12 meters
1 chunk = 2K
10 GB = 5,242,880 chunks = 2048x2048x2048 meters

How many people can create this amount of voxel content in 10 minutes? I have no idea, but I bet it will be an entire army. At this point the DB is the least of your concerns. The bottleneck is in the network.

The twist comes now: While this rate was sustained for 10 minutes, it was not meant to push the system to the limit. The DB process CPU usage never went up 1% and the memory usage for the process remained at 3 MB. The system was responsive and usable (well as usable as a six year old PC can be), showing no big difference in behavior.

Here is some evidence:

For most of you who are more artistically or design inclined this is certainly the most boring screenshot I have ever posted. But if you are into programming this kind of thing, this is process porn.

Of course the system is doing real work. The main clue is in a different column not displayed by Task Manager: Virtual memory, which was hovering all the time below 20 Megs. Even then the virtual memory was lower than what Google Chrome was using, which was a whooping 99 Megs.

The voxel database is so fast because it uses the same virtual memory management of the OS. So, instead of writing to files in the HD directly, all the information is mapped through the OS paging system. Only the pages that need to be altered go into memory. Also the system does a lazy write to the HD. Even after the process is gone, the OS continues to save the changes to disk.

I feel this is the stepping stone for great things. It will be fairly easy and inexpensive for people to set up their own servers. They could be hosting a lot of players and barely take a hit for it. This of course depends on how the networking is implemented, which leads into another favorite topic of mine: how to make a server that will not bring your PC to its knees. I will be covering that in the near future.

Melodive and Oveja

I couple of days ago I got this game from one of the readers of this blog. It is all procedural, including the progression of music. The name is Melodive and it is available for iOS.

Even without the help of mind altering substances, the game takes you to a different dimension. I did find the control scheme a bit frustrating, but it seems there is a whole class of games using this form of tilt, rotate interface. To many of you out there the controls may seem standard.

Oveja

And there is this other non-procedural game a Russian friend did from scratch, including programming, graphics and music. He named it Oveja, which in Spanish means "sheep". (Why a Russian guy is giving Spanish names to his games, it is beyond me.) The game is fun and equally surreal, but on a different level.

It runs in Android and it is available free from Google Play:

https://play.google.com/store/apps/details?id=com.mastercluster.oveja

If you like air-traffic control games you should give it a try. It is like these games, but with sheep in it. I did not get why the black sheep needed to be segregated from the white sheep. This is 2013, those times should be over.

The next version is rumored to include sheep poop and other equally interesting gameplay mechanics.