I've been continuing my work in DirectX 10, and it has been relatively slow progress, but progress nonetheless. I managed to render the simple character model that comes with the DirectX SDK using the first pass of my pencil shader. The first pass applies 6 different pencil textures based upon how lit that part of the model is. It is currently using the given texture coordinates of the model, so it doesn't look as good as it potentially could. That's an improvement for another time though.
I have been busy working on getting the second pass of the pencil shader working. It's pretty much just a Sobel filter that is applied to the normal map of the scene. In order to get the Sobel filter working though, I had to first figure out how to do post-processing image filters in DirectX 10. Obviously the first step was to render a full-screen textured quad. In XNA, I "cheated" and just used a SpriteBatch. It's not the most efficient way, but it allowed for the fastest development.
There ARE sprites in DX10, but I figured I might as well do it the "proper" way this time. So, I set up my own vertex format (with Position and Texure Coords), created my own vertex buffer (with the 4 vertices positioned in clipspace), and my own index buffer. I then wrote a simple shader that simply passed the vertices through the vertex shader, and applied the texture in the pixel shader. I fired up the app for the first time and Vista completely froze! Ctrl-Alt-Del didn't work, the mouse didn't move, nothing. So I shut it down and restarted and I got the Blue Screen of Death! I booted back into Safe mode and it said that my video card driver had gotten screwed up. I was able to boot back into Windows regularly the next time without changing anything.
I set a breakpoint in my code and stepped through the Update/Render loop about a dozen times without any errors, so I just let it run free again. The exact same problems occurred again, thus causing a reboot. I had no idea what I was doing wrong to screw up my computer, so I looked at several of the DX tutorials line by line. I realized that I had forgotten to tell the video card what my vertex input layout was. I added in the 3 lines of code to do it, and bam, everything worked! So I now have full screen textured quads working in DX10. An added bonus is since I positioned the vertices in clipspace, then it works on any resolution without requiring any updates or changes, plus there are no matrix multiplications that are needed to be done.
The next task I have to work on is to get mutiple render targets working in DX10. I need this because as I mentioned earlier, my goal is to render a normal map of the scene, and then pass that normal map into the Sobel filter shader.
