Version 0.14 Teaser and Future Plans

I’ve discussed most of this on the DaggerXL forums already, but I’ve decided to make a full post in order to inform everyone else as well.

Version 0.14 Teaser

The build has had some delays, due to issues with time not technical concerns. Anyway the build is right around the corner – here are some teaser screenshots of a different climate (the desert). All regions and locations will be available in the upcoming version, including the Sentinel region shown below. I’ll talk about the terrain system more in the next blog post, but you can see the smooth terrain transitions between terrain and locations in the screenshots below, especially in the towns. This is accomplished by smoothing out the terrain around the locations, interpolating between the location height and the desired terrain height with a fade off based on 2D distance from the location boundary. In addition the terrain textures are matched to the edges, when possible. Also some of the flats are scaled wrong in the screenshots, I know what the problem is but haven’t fixed it yet. It will be fixed for the next build.

Sentinel City

Desert Towns

Future Plans

As I’ve discussed on the forums, after this build is released I’m planning on taking a break from the terrain system – roads, rivers, improved water and long distance rendering to the horizon will be implemented later. I will immediately begin implementing gameplay features, as listed in the 0.3 milestone in the Roadmap.

In order to test a variety of features and skills, there will be 4 default characters that you can choose from – starting in version 0.15 and beyond. These will include a mage type class, fighter type class, assassin and generalist. In addition the quick save will be extended to save the inventory, stats, skill levels and character level. As discussed in the Roadmap leveling will be implemented, proper skill use, spell system, ranged combat (bows), the remaining base enemies, proper loot, the remaining weapons and armor, the rest of the dungeons and more. Once the gameplay features for version 0.30 are complete I’ll finish up the terrain while people test out the everything else.

Stay tuned, there will another blog post in the very near future when version 0.14 is released!

 

Terrain Texturing, Foliage and Collision

It’s been a while since the last post, I went for about a week or so without being able to work on DaggerXL very much. Since then there has been some more progress on the terrain system.

Foliage

The foliage positions are now generated directly from the terrain vertices. The effect of this is that foliage no longer shows up in location blocks, like it did sometimes before. It is also generated at the correct height now. You can see the effect in the screenshots below. Look at the hill in the background.

Locations & Collision

As before locations are still flat and the entire location is placed at the global altitude map height for that map pixel. This means that ground collision and Flat/NPC heights are adjusted accordingly. In addition the terrain now “connects” to the locations so that it is continuous. Right now it can be somewhat jarring, but I plan on smoothing out the transitions so that it looks natural. When the player walks on the bumpy terrain, the collision now works correctly. Rather then colliding with the polygons themselves, when terrain tiles are generated around the player the heights are saved. To figure out the height at the player’s position, the engine figures out which quad the player is standing on and determines the height using bilinear interpolation.

Terrain Texturing

This turned out to be more of a task than I originally thought. Most engines have a few repeating textures and then some global blend maps (or vertex blending) to blend between them. Daggerfall had 56 tiles, all derived from 4 basic terrain textures (water, grass, dirt, rock – for temperate climates). The first thing DaggerXL does it pack these textures into an atlas for each climate. The engine automatically generates a small border around each texture by extruding the edges, so that filtering works correctly. As you’ll see later, I have to make the border larger, there are still artifacts (edges visible around the quads) at lower mip-levels.

Here’s the atlas generated for temperate terrain:

In order for each quad in the mesh to map to a different tile, extra vertices have to be generated (4 per quad) – though this doesn’t affect the polygon count or index buffer size. Anyway when each terrain tile is generated, a 16×16 tile index is generated, from which the texture coordinates are derived.

A screenshot of random texture tiles applied, just to verify the texture coordinates are generated correctly:

So the next step is to choose one of the base terrain tiles for each vertex, from which the final tile will be generated. The terrain tile index is generated from coherent noise, with probabilities generated based on altitude and proximity to the shore line. This means that each terrain cell can be generated independently and the textures still match up, as you’ll see later.

A screenshot of the base texture tiles being generated with no transitions:

If you look at the terrain atlas again, you’ll notice that most of the tiles represent transitions between different base tiles. For each quad, the base texture tiles generated for the 4 vertices making up that quad are used to determine the proper transition tile. Using Marching Squares, for a transition between 2 different tile types within a quad, there are 16 transitions (lines or curves if generating an isoline or isocurve). In Daggerfall, to preserve space, those transitions are generated from a much smaller set of tiles using horizontal flipping, vertical flipping and 90 degree rotation. So I pack the vertex base tile types into an unsigned long which then accesses a table that specifies the tile index, 1 bit for u flip, 1 bit for v flip and 1 bit for 90-degree rotation. The following screenshots show this off, however only transitions between grass and dirt have been finished, though the rest will be in soon.

I have to finish the terrain transition table, handle texturing along coastlines and then I can release the next version.

 

Terrain Work Continues

The work on the terrain system continues. Each terrain “tile” (16×16 quads) is currently textured based on the climate derived from the climate map. Since the climate map is only 1000×500, each tile is actually only a small part of the pixel – which makes the climate and texturing very coarse right now. However I can travel throughout the whole world and the terrain textures and foliage change properly based on climate.

All the screenshots show the world map, with climates color coded. The white dot is the location of the player.

Below you can see 2 of these climates:

Note that some of the plans aren’t properly scaled yet, fixing that is on my to-do list.

In addition I’ve started incorporating the altitude information to generate varying heights. The base height is generated from a specially filtered version of the 1000×500 altitude map, where the individual vertex height is calculated from the neighboring large scale heights using bi-cubic interpolation. Heights tend to change slowly, for the most part, using this large scale map so I also add some noise to this base height. This noise is scaled based on the overall altitude generating larger perturbations at higher altitudes and smoother terrain near the coasts. Obviously this must be generated at run-time, whenever the terrain tiles in view are updated, so the noise uses the parametric tile interior as the fractional part and the tile location in the world as the integral part. This means that the noise is generated independently for each terrain tile but it still lines up without having to do any post processing. There are still a lot of work to do, such as making sure plants are on top of the terrain (instead of at height zero) and collision detection. In addition there are some seams due to the method of generating the normals but that will be fixed soon. Also note that the polygon density for this terrain is exactly the same as vanilla Daggerfall. However the noise is more coherent (multi-frequency Improved Perlin Noise), the view distance is further and gouraud shading is used instead of flat shading. I will implement a version that is more like the original terrain, with flat shading and simpler height changes later, for those that want a more authentic experience.

Here’s a sneak peek at the bumpy terrain:

Work continues but the terrain is coming along nicely.

 

Terrain Work Begins

I have begun work on the terrain system for DaggerXL. I read and parse the WOODS.WLD file and the CLIMATE.PAK file.

WOODS.WLD stores the global altitude map as a 1000×500 grayscale map. This will be used as the global, large scale altitude for the terrain. The lowest altitudes indicate where water exists. Obviously the real height in game for smaller bodies of water need to be determined by the height of the surrounding terrain.

This file also contains the “noise” data which determines the roughness of the terrain (how much the terrain undulates), which I’m storing but not using yet. Daggerfall natively doesn’t appear to use the altitude data but instead the noise is scaled by the altitude. So instead of generating large scale altitude changes, Daggerfall generates larger undulations about the zero plane. DaggerXL will also support this, if you want true Daggerfall emulation. However, DaggerXL will use the altitude map for large scale height and then modify that by the noise, scaled to reasonable values, by default.

The CLIMATE.PAK file contains the climate data for the world. This determines which texture set is used for the ground, what sky textures to use and what the foliage looks like. It will also modify the textures used for exterior, interior and dungeon blocks. Here is the climate data render based on the climate type. Note that I colored the water areas blue, although the data in the Climate.pak is one of the 4 main climate types here too.

To get a more intuitive view of the terrain shape, I generated a Normalmap based on the Heightmap and apply some simple lighting. So smooth out the 8 bit height transitions, I apply a filter where the strength of the filter is based on the altitude of the center pixel – so that the coastlines are preserved. This was done at runtime with shaders, it’ll look much nicer when everything is combined – including high resolution details and better smoothing (fix up coastlines, raise lakes to the surround terrain instead of having them generating giant sink holes, etc.).

I’ve also started to use this data in the world itself, though at the very coarse 1000×500 scale right now. You can see a crude coastline in the screenshots below. Please note that this is still using 1 texture per terrain cell and is very, very coarse. But it’s just the beginning.

 

“Pre-Alpha” (previously Demo 2) Released!

This is the last monolithic demo, which is why I’m now calling it the “Pre-Alpha”. From now on, builds will be incremental – smaller and more frequent. Many things have been added since my last post, including simple fast traveling, levitate, horse travel – implemented as a fast run for now, a title screen, interior doors with sound effects, dungeon music, a cheesy quick save/load and more. For the next incremental build, I’ll start working on real terrain system – so expect more exciting updates.

Note that the dungeon gameplay hasn’t changed much, not all the sound effects have been implemented – though the system is in place, terrain is a flat plane for now, NPC’s sometimes go through buildings (I have to offset their movement grid still)… this is very much a work in progress.

Don’t forget to look up the controls for casting levitate, fast travel, quick load and quick save. Also remember that the travel mode button on the full UI works as well.

Visit the downloads page to get the latest build.

The build is now ready for download!

**Update**
The build has been updated to version 0.13.
Version 0.11 removes the ‘F’ and ‘G’ debug keys, as was originally intended, and adds support for texture replacer mods.
Version 0.12 fixes a texture replacer bug dealing with indices. Use this version if you want to see mods.
Version 0.13 fixes a crash when hitting “I” (which is no longer hard-coded). Also fixes 320×200 mode so it works in exteriors.

 

Walking NPCs, Climbing, Sky Fog – Pre Demo 2

I’m pretty close to releasing Demo 2, I’ve implemented the last features – now it’s just a matter of fixing a few bugs. This post has a movie showing various features at the end.

Properly spawning and walking NPCs

I’ve had exterior NPCs in before but they didn’t move or spawn properly – sometimes they’d come in at invalid spots. I decided to try raycasting to determine if a spot was valid but there were multiple problems with that. There would be a lot of raycasts and collision checks every frame as NPCs were added, removed and move. In addition, the raycasts would determine a spawn to be valid but it really isn’t, it’d be in a bad area or too close to a building. To fix that would require multiple raycasts to cover an area or other more costly solution. Instead, I gave up on the raycast / collision idea and picked a simpler solution. When a block is loaded, each vertex on its ground plane (16×16) is considered a valid spawn point. Then right after load (this is done only once) every building is checked – any spawn points inside it’s bounding box is considered invalid. Each row of 16 points is stored in a bitfield – so there is an array of 16 of these bitfields, one per row. So now a grid is generated around the player, where each vertex on the grid is a valid spawn point or not. When spawning a new NPC, it randomly picks a valid spawn point that is outside the field of view and within a certain range of the player. The beautiful thing about this system is that the NPCs use the same grid for movement. The NPC randomly picks a destination valid grid point from the 8 vertex region around its current point and that is used as a target point for the NPC to move towards. Once it reaches the new grid point it then selects another one and moves towards that. Full movement without any collision detection – except for guards when they are chasing the player, which act like enemies (these will not be in the demo). Finally when an NPC is outside a certain range and not in the player’s field of view, it is recycled – it respawns at a closer valid point and changes NPC skin.

Here you can see some screenshots of the NPC grid, green points are valid spawn points, whereas red points are outside of range or invalid.

Climbing

I implemented the first pass of the climbing ability, although it’s a little fickle still. Anyway when the player collides with the world, if the “final” slide plane (i.e. the plane of the collision polygon(s) relative to the player velocity) causes the player to stop moving on the first collision iteration is purely vertical (i.e. the “up” component of the normal is 0.0) then the player can potentially climb that surface. Once the game decides that the player can climb, decided based on skill eventually – there is no skill check yet, then gravity is disabled and the player moves upward when pushing into the wall. As soon as the player stops moving forward or moves past the vertical surface, climbing mode ends and regular gravity is activated. It still needs some work but you’ll see climbing in action in the movie.

Sky Fogging

The fog color now matches the sky color, as you’ll see in the movie. This is done by rendering the sky background into a relatively low resolution texture. The vertex shader projects the object’s vertex position and generates texture coordinates into that texture. The color from that position is then used as the fog color, so objects will appear to “fade into the sky” no matter where they are.

There have been other miscellaneous tasks completed and issues fixed, but these are the major items.

The Movie

In the movie below, the player starts in a small town roughly South of Daggerfall. The player runs through the woods and you’ll notice that the walls of Daggerfall (and a few tall houses) are visible the whole time even though it takes about a minute and a half to run there. Various interiors are shown as well as NPCs walking around. You’ll also notice the smooth sky transitions as the time of day changes. There are some bugs and I accidently bring up the inventory in part of the movie, but it’s coming along.

done

High Resolution Movie

 

Daggerfall At Night.

Lire ce message en français
Przeczytaj tę wiadomość po polsku

Work on the demo continues. This post does not have any new screenshots but there is a movie. You can click on the high resolution link below the movie to download a higher resolution version. Much of the work was in optimizations, music and sound – hard to show screenshots of that stuff.

Music & Sound

Midi music and sound effects have been implemented, though some of the midi loops a little roughly, which will be fixed later. Note that at night you can hear the crickets (though it’s harder to hear in the video then while playing the game for some reason), the cries for vengeance as well as the appropriate night time music. It also switches music tracks based on the interior type as well, you’ll see this when the player goes inside a tavern in the movie. In the next post I’ll put a movie showing more of the wilderness (during the day) and more building interiors.

Mesh & Flat Rendering

(Much of this has been discussed on the forums already) Everything is rendered using a deferred “render queue” system, basically each mesh material or flat is submitted as a “entry” and then all the entries are rendered for a given frame. There are multiple lists: opaque materials with base lighting (character light, directional light outside, fog), opaque materials + points light passes (2 lights per pass right now) and translucent/transparent materials (flats). [Note: only those materials that are not culled get submitted. There is a hierarchical culling system not mentioned here as well as other systems such as contribution culling]. I will probably add the z-prepass and light stenciling soon, these are pretty easy with this system.

I’ve added sorting to this as well. So opaque materials are now sorted by shader (right now each list only uses 1 shader), then by texture, vertex buffer then index buffer. This reduces the state switching as much as possible.
Transparent materials are now sorted based on distance to the camera plane, which means that they are composited in the correct order – so no more weird edges around sprites (unless they are built into the texture itself). The only exception to this, is emissive flats such as streetlights. There is still an error, due to the way the emissive setting is handled, that causes dark edges around the flat. This will be fixed.

I also fixed various other issues as well, such as regenerating plant distributions more often than necessary.
There are still some improvements to be had here, but with the sorting it all looks right at least.

In the video below, you can see me walking around Daggerfall city at night. You’ll notice the sound effects, music, sky transition from day to night, bloom, interiors with lighting and interior NPCs. Note that there are some errors that need to be fixed:

* Dark edges around emissive flats (discussed above).

* Fringing around the edges of the screen from the bloom.

* Flickering/strobing of the bloom effect, this will be fixed before the demo.

The movie:

done

High Resolution

 

NPCs, Regional Interior Textures, Windows and Bloom

Interiors-
This time around the interiors were almost completed. Now NPCs show up where they’re supposed to, you can even click on some of them – although right now you can loot them like dead bodies – except with no loot. Obviously they’ll talk eventually, though not in the next demo. Next up the interior textures are now corrected for region like the exteriors. The taverns, mage’s guild, temples, etc. look more fitting for the Daggerfall region now. The doors that connect the exterior and interior are fully functional, before they only worked if you were on the ground level. Now you can go through upper level doors, climb the spiral staircase and get on the wall and so on. And finally the interiors use the proper dungeon like lighting (no more directional lighting) and the lights work correctly now.

Exteriors-
Exterior NPCs are now showing up in cities, although the work is still preliminary. The outdoor lighting intensity also changes with the time of day, so the lighting matches the sky better. The sky now smoothly fades between images, so you won’t see a pop when it changes. This cross fade happens over about 5 seconds, so the gradients actually change very smoothly. I also implemented an optional sky filter that smooths out the sky gradients and dithering but preserves the edges – the sky looks smooth but not blurry. This is done on load, if the option is enabled, so there is no additional runtime cost or memory consumption – so there’s no reason not to use it unless you don’t like the effect. The luminance range is computed for each image and used to automatically adjust the filter parameters to preserve edges and provide just enough smoothing regardless of how bright or dark the image is.

Windows now work correctly. No additional memory or textures is used and the palette information is lost once the textures are uploaded to the GPU, so it took a little work to get the proper color to change without any filtering artifacts and only changing the correct color and nothing else. The way the renderer works is that alpha controls opacity and emissiveness. However with outdoor textures, not including flats which have a different shader, no pixels need to be emissive except for the windows at night. So what I did is leave the color alone (a middle-dark gray) and set the alpha to fully emissive. Then in the exterior shader (for geometry) I blend in the proper window color over an alpha range that is otherwise unused, which gives a smooth bilinear look even though its blending in a different color. I also blend between different colors for the window based on time of day (represented as a single float in the shader), as well as different final emissive values. This way windows are full bright at night but are properly shaded during the day.

This leads to the last feature I want to talk about, an optional feature: bloom. Basically only those pixels that are marked as fullbright (windows when they’re fullbright at night, light source flats) glow and only if the intensity of the color is greater than a threshold – that way fire on top of a torch will glow but not the wood. The geometry is rendered only once, the intensity value is written into a second render target as the scene is being rendered. You can see the bloom in the screenshots below, they are the last set.

Interior NPCs, proper region texturing and proper interior lighting:

Exterior NPCs

Exterior Windows

Bloom

 

Time, Sky and Interiors

The next thing I implemented was proper time of day support. The city lights turn on and off based on the time and the lighting/fog get much darker at night. This leads to the next visual feature: skies! During the day, Daggerfall uses 2 textures for the sky – an East and West image, which form a sort of cylinder around the player. For a given region there are 32 different sets sky images, ranging from dusk to noon which are then played backwards from noon to evening – but with the east and west textures swapped in order to have the sun travel the whole arc through the sky. As you look up, DaggerXL limits how far up it scrolls the sky so that you can’t see past the top (which wasn’t a problem in Daggerfall since you couldn’t look up as far). No such limits are used when looking down since the bottom color is stretched as far as it needs to be. Night skies are simpler, there is only one texture that is tiled to form the “cylinder.” [Note: it's not really mapped to a cylinder because that would imply perspective distortion when looking up or down but its an easy way to visualize the effect]. I also, finally, adjust the exterior building textures based on region so all the buildings should now look correct from the outside.

In addition, if a building has an interior you can click on the door and enter. Interiors show up correctly, except for modifying texture indices based on region in some buildings, as do all the decorations and flats. The only things missing are doors (the internal doors that rotate, the doors leading outside are there) and NPCs. Those things are next on my list. Anyway you can also leave and go back outside too, of course. When you go inside, the exterior is turned off and only the proper interior (and it’s flats) is rendered. Conversely the interiors are not rendered when outside.

Sky screenshots at different times of day:

Screenshots of various interiors:

 

World map, flats, objects and foilage…

A lot of progress has been made when it comes to rendering the exterior world of Daggerfall.

DaggerXL now supports the seamless world map. The way it works is that the game tracks which map cell the player is in (a map cell is the same size as a City block, so for example Daggerfall is 8×8 cells) and the player’s coordinates within that cell. DaggerXL reads the location data from disk and stores 2 different hashes which are keyed from the map tile coordinates, one stores the location data – the location name, number of blocks, name of the blocks, etc. and then one stores the loaded block data itself (which I call “tiles”). As the player moves and changes tiles, the relative coordinates are adjusted accordingly and then the tile locations are updated. New tiles are loaded or unloaded as necessary based on the location map, and neighboring tiles are rendered or collided against based on the tile map (which has loaded tiles only). In this way tiles are loaded from location data as needed, the coordinate system is tile relative so there are no precision issues regardless of the size of the world, collision only occurs on neighboring tiles and terrain is rendered everywhere where no tile exists (or is not loaded yet). So I can travel around the Daggerfall region and go to any location. All the cities are showing up as well as ruined castles, temples, graveyards, dungeon entrances and more.

Secondary objects and flats are now being rendered within city blocks. Things like wagons, horses, signs, fountains and other decorations. In addition, during night times hours, lighting is working on the appropriate objects. Unlike, Daggerfall however, the lighting effects the ground and terrain. In addition, even though the environment gets dark around the player at night – just like Daggerfall – lights can be seen from a long distance.

Finally foliage is also supported, as seen throughout the cities. Exterior terrain tiles also get random foliage, where 1 tile away from a location the density is 30% of normal (to ease in from no extra foliage to full forests). The random seed is tied to the actual tile location in the world, not the relative location, so you’ll also see the same plants in an area even though it’s random. Of course this is just a quick first pass – the demo following this one will have better terrain and foliage distribution.

Finally the screenshots:

Nighttime lighting (before foliage was implemented):

Foilage, flats and objects:

If you look closely, you can see a town in the distance through the trees…