tag:blogger.com,1999:blog-3779956188045272690.post6187276015564262258..comments2024-03-18T03:21:55.239-04:00Comments on Procedural World: A trip to the Voxel FarmMiguel Ceperohttp://www.blogger.com/profile/17586513342346629237noreply@blogger.comBlogger19125tag:blogger.com,1999:blog-3779956188045272690.post-1701304552342234392013-09-12T06:52:13.277-04:002013-09-12T06:52:13.277-04:00Are you doing any culling of the meshes generated ...Are you doing any culling of the meshes generated by multi-layered volumes when parts get occluded?Reed G. Lawhttps://www.blogger.com/profile/17192175531543923780noreply@blogger.comtag:blogger.com,1999:blog-3779956188045272690.post-43040761574389891862012-06-03T22:36:25.222-04:002012-06-03T22:36:25.222-04:00I guess I didn't understand the sense of scale...I guess I didn't understand the sense of scale you are creating.<br /><br />I might suggest a few posts with something along the lines of a world map, it would really help illustrate the sheer level of detail you are creating. (perhaps you've done so already? I haven't gone through all the posts yet)Anonymoushttps://www.blogger.com/profile/11800572223165657557noreply@blogger.comtag:blogger.com,1999:blog-3779956188045272690.post-47053279630893223502012-06-03T19:41:07.911-04:002012-06-03T19:41:07.911-04:00Correct, I do not care much about players modifyin...Correct, I do not care much about players modifying the game world at this point. The goal of this project iteration is to have good looking content without requiring much effort from game designers and artists.<br /><br />The benefits of voxels still apply in this context, only for the game maker. The main advantage they bring is you can easily combine layers of data from very different algorithm and still obtain clean watertight meshes.<br /><br />Dual contouring is quite fast and could be performed in real time. The actual procedural generation is what takes time, of course depending on how much detail you want. If you are doing many layers of terrain, forests and intricate cities, and you want to have several kilometers into the horizon, the current generation of hardware cannot cope.Miguel Ceperohttps://www.blogger.com/profile/17586513342346629237noreply@blogger.comtag:blogger.com,1999:blog-3779956188045272690.post-4324414556884611952012-06-03T19:02:35.565-04:002012-06-03T19:02:35.565-04:00One of the benefits I see in voxel data is that it...One of the benefits I see in voxel data is that it is deform-able/malleable in the game. Judging by this post, it seems you have sacrificed that aspect for speed?<br /><br />If that is the case what benefit are you getting out of voxels?<br /><br />Also, I was under the impression that dual-contouring be done in real time, why do you need a render farm?Anonymoushttps://www.blogger.com/profile/11800572223165657557noreply@blogger.comtag:blogger.com,1999:blog-3779956188045272690.post-24667598793524950632012-02-06T12:53:02.690-05:002012-02-06T12:53:02.690-05:00For realtime is just polygons. Grass, flowers and ...For realtime is just polygons. Grass, flowers and foliage uses instanced billboards. Small details like pebbles and little cracks is in the textures. You can see severals posts I did on those topics at the end of 2010.Miguel Ceperohttps://www.blogger.com/profile/17586513342346629237noreply@blogger.comtag:blogger.com,1999:blog-3779956188045272690.post-67705681198587016452012-02-06T12:44:39.753-05:002012-02-06T12:44:39.753-05:00Thanks again for your time. So the farm auto edits...Thanks again for your time. So the farm auto edits "levels" via parameters. What's the realtime engine instead? You said you run it at 60fps. The compression is good at LODing the geometry, but so it's not stored as voxel, if by voxel you mean a "[x][y][z]" matrix, what do you use instead. While lodding is good to streamlin the contour of massive surface to sum the noise, how about grass, grain fields an other such as fine and spiky details? Crysis is good at instancing millions of them just as a visual effect, just to say, even if the actual geometry of the land is actually low res and dull.<br />I have now present the big hassle that fragmentation could cause, two pieces of data have free space in between then, suddenly you need to allocate some space but it's bigger than the free space available between all of them. The ideal is to have safe big chunks of space allocated, and simply swap the objects the same space is representing, but I'm not sure. Is this a problem with octrees more than with uncompressed space? Hey you really got me to think more about this issue about my possible idea, thanks.<br />AntomeAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-3779956188045272690.post-65907090195301754312012-02-05T22:22:45.421-05:002012-02-05T22:22:45.421-05:00Note I never store voxels. Currently I have three ...Note I never store voxels. Currently I have three different entities in the procedural world: terrain, trees and architecture. Each one is represented in very memory efficient forms. The contouring phase takes these abstract definitions and outputs a high detail polygonal surface, which is later compressed for realtime rendering.<br /><br />By garbage collector I meant something like Java's garbage collector.<br /><br />Memory fragmentation is a nasty effect found in default heap memory managers. If a program runs for a very long time and performs a lot of memory allocation for blocks of different sizes, even if there are no leaks in the code, you will end of with a lot if smaller sized available blocks. Even if they add up to a lot of memory, a large block cannot be allocated anymore. This is similar to what happens in hard drives after you have deleted and created many files. You need to run a defrag tool so your HD is fast again. With memory it is worse, instead of running slower memory allocation just fails despite the fact there is enough memory.Miguel Ceperohttps://www.blogger.com/profile/17586513342346629237noreply@blogger.comtag:blogger.com,1999:blog-3779956188045272690.post-48161052509301401612012-02-05T20:01:56.531-05:002012-02-05T20:01:56.531-05:00Sorry, I'm italian, I meant seafight, not batt...Sorry, I'm italian, I meant seafight, not battle ship.<br />And obviously by distorting these voxel maps of curves the polygon is used as a reference, maybe some materials can be generated along these surfacese, like brick or wood planks.<br />AntomeAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-3779956188045272690.post-73421592218681139312012-02-05T19:53:45.014-05:002012-02-05T19:53:45.014-05:00Awesome reply, thanks.
Well, I read this is a farm...Awesome reply, thanks.<br />Well, I read this is a farm so a voxel render farm, not like Pixar render farm, but to bake complex procedural graphic solutions via parameters, getting the most natural graphic results, right? So the realtime engine-editor would be a separate subject, then it would make sense to "compress" (like most realtime poly engine do via poligons)using a mix of procedural and raster graphic. Example, you use something that would usually be a handful of polygon tiles, then along this reference you generate the voxel (also just heightmapped) approximation of convex and concave spline, to generate a soft mound, you don't have to store so much voxel mode because they get rotated and affine distorted to blend together. Still in an octree or something similar. That is just an idea, I admit I didn't code much more than a battleship with ship instancing, lol :D. But i pictured that simple procedural solution can be used in realtime, I think at how fast were the Unreal procedural water back in '98 or the displacement mapping now (though lame at times).<br />I agree about wishful thinking, by garbage collector you mean something along the line of swapping the same memory location for, like, different nodes, when others are far away or compressed, or view portals or Pvs?<br />And about memory fragmentation you mean by un unfragmented having all nodes in order of position, allocated in memory close together, but at the cost of not being able to move them real time?<br />AntomeAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-3779956188045272690.post-13128515010627440562012-02-05T18:08:59.217-05:002012-02-05T18:08:59.217-05:008 GB RAM is the minimum requirement for the machin...8 GB RAM is the minimum requirement for the machines I'm selecting for the farm. This does not mean a farm node cannot run on 4 GB for instance.<br /><br />That being said, memory is cheaper than time. Making a solution slower just to save memory does not make sense to me. Time is money.<br /><br />In a farm node, most of the RAM is taken by the contouring process. My focus was speed, so I had to make sure default memory management would not get in the way. Even if the contouring is done adaptively over an octree, I assume the worst case scenario from the onset. I wrote the memory management logic to be aligned with the octree and the contouring algoritm that runs using OpenCL. In most cases a large percentage of the reserved memory is not used, but when bad cases appear, the system is able to deal with them without a hitch. This is of course at the expense of having a large pool of memory locked all the time. Again, since my top concern is speed, it pays.<br /><br />I love this topic and I wish I could spend some time on it.<br /><br />It is common for programmers to adopt a wishful thinking stance when it comes to memory management, a garbage collector being the worst case, then we are surprised when our solution is performing poorly due to unpredictable hikes from memory fragmentation, swapping and thrashing. Memory management is an integral part of any algorithm. In many cases you can afford not to care, a garbage collector will do great. In other cases it will be the core of everything you do.Miguel Ceperohttps://www.blogger.com/profile/17586513342346629237noreply@blogger.comtag:blogger.com,1999:blog-3779956188045272690.post-51843559569231488122012-02-05T15:36:14.580-05:002012-02-05T15:36:14.580-05:00Cool, but wait, why 8 Gigabytes of Ram just for mi...Cool, but wait, why 8 Gigabytes of Ram just for minimum requirement?<br />Being this procedural I'd have said much of the burden lied onto cpu cicles.<br />Couldn't be there a system that swaps according to the moment the advantages of storaging and the sparing of procedurals, maybe using reference points like polygon and across them, something like a voxel approximation of a nurbs (a 3d bezier curve also called spline), in place of the texures different leves of noises an displacement + details "tissue crease" effects, etc. to simulate mountains or water and other curves and effect depenting on what you wish.<br />Just a thought, because this blog hooked me.<br />bye and thanks.<br />Antome (couldn't log with yahoo)Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3779956188045272690.post-58234274972385663132012-01-20T22:31:13.105-05:002012-01-20T22:31:13.105-05:00@dehaul: Thanks, but don't be fooled by the sh...@dehaul: Thanks, but don't be fooled by the shinny stuff. I have not spent much time or energy on this. I have been reguritating the same algorithms for a long time. If you check the very first video I posted in this blog you will see pretty much everything was already there.<br /><br />My goal is to create a game to showcase this system, sell it and hopefully license the technology. But I'm progressing so slowly any novelty in what I do will be gone by then.Miguel Ceperohttps://www.blogger.com/profile/17586513342346629237noreply@blogger.comtag:blogger.com,1999:blog-3779956188045272690.post-52561306434904605902012-01-20T17:59:14.617-05:002012-01-20T17:59:14.617-05:00That is really amazing! It is incredible the amoun...That is really amazing! It is incredible the amount of time and energy that you have put into this... which is only surpassed by the quality of the resulting product.<br /><br />I really want to know why are you are doing this? For the LOLs? To get a job somewhere? For a patent? What is driving you to do this?dehaulhttp://virtualizedlimbo.comnoreply@blogger.comtag:blogger.com,1999:blog-3779956188045272690.post-12620050069137063412012-01-20T16:54:24.091-05:002012-01-20T16:54:24.091-05:00You're doing so much more than 'just' ...You're doing so much more than 'just' creating a virtual world.<br />What you're doing is just fabulous!Tottelnoreply@blogger.comtag:blogger.com,1999:blog-3779956188045272690.post-90713756769572115462012-01-20T09:03:35.372-05:002012-01-20T09:03:35.372-05:00nice :)nice :)Aroidzaphttps://www.blogger.com/profile/06891654803738975116noreply@blogger.comtag:blogger.com,1999:blog-3779956188045272690.post-18684026128382215292012-01-19T23:57:24.917-05:002012-01-19T23:57:24.917-05:00@Scott: It is all in the contouring method you cho...@Scott: It is all in the contouring method you choose. If it recovers sharp features then it can also do smooth surfaces. Do not use the classic Marching Cubes for instance. Everything out of it looks like a blob. Extended Marching Cubes can do sharp features but it is a lot of work for nothing. The method I use is Dual Contouring. You can search this blog for those terms and you will find a post with some useful links, some have working source code.<br /><br />Once your contouring can reproduce sharp edges, any sharp features in the implicit function you use will produce meshes with clean sharp edges.Miguel Ceperohttps://www.blogger.com/profile/17586513342346629237noreply@blogger.comtag:blogger.com,1999:blog-3779956188045272690.post-10889686840516177902012-01-19T21:26:25.374-05:002012-01-19T21:26:25.374-05:00Very interesting topic! I love it. Something I'...Very interesting topic! I love it. Something I've been struggling with a lot myself recently has been how to generate the surface mesh and be able to tell the 'contour' system to generate parts of the mesh as smooth / rounded and parts as sharp. Example: A flat voxel surface with only a few voxels above the surface. Is it going to be rendered as a smooth mound or a small cliff face? How do you determine what's a cliff, and what's a rolling hill?Scott Richmondhttps://www.blogger.com/profile/11435228878954774684noreply@blogger.comtag:blogger.com,1999:blog-3779956188045272690.post-7969536026717129202012-01-19T18:12:33.122-05:002012-01-19T18:12:33.122-05:00@Justin: ZeroMQ is alright, but I don't really...@Justin: ZeroMQ is alright, but I don't really have any problem that cannot be solved with traditional sockets. You would be surprised about how little code in my solution deals with the messaging transport. I would use a library only if it does something I don't know how to do, or that I cannot learn how to do. Even then I would loose sleep over the fear of ending with a dead horse attached to my carriage.Miguel Ceperohttps://www.blogger.com/profile/17586513342346629237noreply@blogger.comtag:blogger.com,1999:blog-3779956188045272690.post-69298508472301854442012-01-19T17:25:42.258-05:002012-01-19T17:25:42.258-05:00Have you looked at ZeroMQ? For this sort of messag...Have you looked at ZeroMQ? For this sort of messaging-passing architecture, it's a godsend. Obviously you already have a working solution, but if you do a rewrite, ZeroMQ may be able to save you a lot of time.Anonymousnoreply@blogger.com