Game Studio design, and code

Since the game studio is the primary interface for game development for the engine, a lot of thought and planning is required to make it as useful as possible.

First, a screen shot:

Now, the nuts and bolts.

The main form has a menu, tab control, and frame.  The frame (on the right side) will hold the project files in a tree view.  All script files will be listed there.  The tab pages will be where project files are edited.  The File menu allows projects to be opened, saved, closed, and printed.  The Edit menu contains standard cut, copy, paste, and select all commands.  There will be more menu items added for different file types.  I'm going to add in code compiling and testing functions soon.

There are different types of tab pages.  The one that is visible is a code editor tab.  All of the UI controls are UI members of one custom UI object, CodeControl.  The HScroll bar, VScroll bar, and Combo box are just regular controls as members of the custom control.  The text and line numbers are being drawn onto a customized group box object.  The custom group box object code is not very complicated, it just modifies some of the functionality of the default group box UI object to make it behave differently. 
The code for the custom group box:

Public Class EnhancedTextBox
    Inherits GroupBox

    Public Event DrawMe(ByRef e As System.Windows.Forms.PaintEventArgs)

    Public Sub New()
        MyBase.New()
        Me.SetStyle(ControlStyles.AllPaintingInWmPaint Or ControlStyles.UserPaint Or ControlStyles.OptimizedDoubleBuffer, True)
        Me.UpdateStyles()
        Me.Cursor = Cursors.IBeam
    End Sub

    Protected Overrides Function ProcessCmdKey(ByRef msg As System.Windows.Forms.Message, ByVal keyData As System.Windows.Forms.Keys) As Boolean
        If keyData = Keys.Up Or keyData = Keys.Down Or keyData = Keys.Left Or keyData = Keys.Right Then
            OnKeyDown(New KeyEventArgs(keyData))
            ProcessCmdKey = True
        ElseIf keyData = Keys.Tab Then
            OnKeyPress(New KeyPressEventArgs(Chr(9)))
            ProcessCmdKey = True
        ElseIf keyData = Keys.Delete Then
            OnKeyPress(New KeyPressEventArgs(Chr(127)))
            ProcessCmdKey = True
        Else
            Return MyBase.ProcessCmdKey(msg, keyData)
        End If
    End Function

    Protected Overrides Sub OnPaint(ByVal e As System.Windows.Forms.PaintEventArgs)
        e.Graphics.FillRectangle(Brushes.White, Me.ClientRectangle)        'clear client area
        e.Graphics.DrawRectangle(Pens.DarkGray, Me.ClientRectangle)        'draw boarder
        RaiseEvent DrawMe(e)        'call the draw event
    End Sub

End Class

The code for the custom rendering is a part of the CodeControl custom UI object.  I wish I could list all of that code here, but it is about 950 lines of code long.  Also, the code is not complete yet; there is a lot of functionality that I need to write.  When I get more of it done, I want to make a post devoted to covering the ins and out of how it works.
The brief version is the CodeControl uses the KeyPress, KeyDown, MouseDown, MouseMove, and MouseUp events from the EnhancedTextBox object to capture user input, and manipulate text that it stores in an array.  The DrawMe event triggers rendering code on the screen, using values from the HScroll and VScroll to determine what text to draw.  A syntax processor sets the color on recognized keywords, double quote delimited text, and numbers, that the DrawMe event uses to render colorized code.  I created overrides for the Text and Font properties of the EnhancedTextBox to handle getting and setting text from the control, and setting the font to be used for rendering.

More to come later...

VB.net, Getting arrow key input on controls

If you haven't already, you should read my previous blog post on Custom Text Boxes in VB.net, otherwise this post might not make much sense.

So, I got most of the functionality of a good text box working today, except using arrow keys to move the caret around the text.  I tried many different approaches, but no matter what I did, arrow keys and tab were moving focus around in my form instead of being handled by KeyDown.  I went back into the EnhancedTextBox class and started to poke around the protected overrides functions to find one that might help me out, and boy did I ever find it: ProcessCmdKey.

I added this code to the EnhancedTextBox class:

Protected Overrides Function ProcessCmdKey(ByRef msg As System.Windows.Forms.Message, ByVal keyData As System.Windows.Forms.Keys) As Boolean
    If keyData = Keys.Up Or keyData = Keys.Down Or keyData = Keys.Left Or keyData = Keys.Right Then
        OnKeyDown(New KeyEventArgs(keyData))
        ProcessCmdKey = True
    ElseIf keyData = Keys.Tab Then
        OnKeyPress(New KeyPressEventArgs(Chr(9)))
        ProcessCmdKey = True
    Else
        Return MyBase.ProcessCmdKey(msg, keyData)
    End If
End Function

Now I'm able to capture KeyCode.Up, KeyCode.Down, KeyCode.Left, and KeyCode.Right in the KeyDown event handler and I can receive Tab as a keypress (ASCII code 9) in the KeyPress handler.  So that's spiffy.
Here's what the text editor control looks like right now.

It doesn't do syntax highlighting yet, but Rome wasn't built in one day, either.

Custom Text Boxes in VB.net

A game engine is only as useful as the tools used to create games for it.  Following this line of logic, it seems obvious that I also need to create a game studio to go along with the game engine.  Aside from creating sprites, animations, and the like, a good game studio is going to have a good syntax-highlighting text editor for writing scripts.  Now, I don't know how many people have tried to created a syntax-highlighting text box before, but I can say that I have, and that it is not as easy as it sounds.  I don't just want syntax highlighting, but also automatic pretty-code formatting.  If you have ever used Microsoft Visual Studio, you already have an idea of what I'm talking about. 

The obvious solution would seem to be a RichTextBox.  We could create an even handler that consumes the OnTextChanged event, and parse the the text in the box.  We would look for tokens and change their color in the RichTextBox, for example, green for comments, blue for keywords, red for text, and so on.  I did just this, using regular expressions to locate keywords, comments, and double quote delimited text, and colorize them.  Unfortunately, this doesn't help with code formatting, only coloring.  It also is SLOW!  Even only handling highlighting on a single line basis, typing even moderately fast would cause the RichTextBox to stop updating text as the regex routine ground down to a near halt.  Handling pretty code formatting is an equal nightmare, as indenting and text replacing kept moving around the system caret (the cursor where you type), and screwing up text entry when typing fast.

So, I've set off on creating my own custom UI control.  I figure since I'm making the whole thing from scratch anyway, I might as well have some fun with it.  I've decided to make the CodeControl a text-box like region with scroll bars (horizontal and vertical), no word wrapping (terrible for coding anyway), and a combo box at the top.  The combo box would populate a list of declared functions that the user can choose from to auto-scroll to the definition of that function.  I thought it would be a nice touch.

To pull this off, we need to consume KeyPress, MouseClick, and OnPaint events.  We need to override any default rendering so we can paint it how we want, including formatted and colored text.  We need to implement our own caret (text cursor), so we can account for it when formatting code.  We also need to handle text selecting, drag-drop moving, copy, paste, cut, undo, and redo for maximum user friendliness.  Oh, and the control has to run fast and efficient. 

To prevent reinventing the UI wheel, it would be nice if we could inherit our new control from an existing UI control.  Text box is an obvious control to inherit from, but TextBox has a system generated caret that we can't (easily) hide and show, except giving and taking focus which messes with our KeyPress handler.  So TextBox is out.

PictureBox is designed to be efficient to draw to using GDI+, and includes many optimizations by default to increase performance.  PictureBox has a major limitation though: no KeyPress event.  There is a KeyDown and KeyUp event, but they don't ever fire because PictureBox is incapable of accepting focus.  So PictureBox is out.

RichTextBox has the same issue as TextBox, so it is out.

Panel can't accept focus either, so it has the same problem as PictureBox.

GroupBox doesn't accept focus normally, but if we call the Focus method from MouseClick event, and watch our GotFocus and LostFocus events, we can appropriately turn on and off our custom caret.  GroupBox doesn't normally accept focus, but for some reason it does have the KeyPress event (I suspect it is inherited from something, and is only there to pass input to child objects.), although the MSDN says that KeyPress never fires, I called Focus, and pressed some keys, and sure enough KeyPress was firing, so that's good. 

So it looks like GroupBox has all the necessary events and functionality we need to implement a new text box, but a group box doesn't really look much like a text box.  That's easy enough to change.  We can use the paint event to Graphics.FillRectangle the ClientRectangle to white, then draw a dark gray rectangle around the ClientRectangle area.  That should do it for looks.  Only trouble is GroupBox is not a control optimized for user drawing.  Anything we paint is going to cause flickering on the control when we refresh, which is going to happen every keystroke, and every 1/2 second for the caret blink.  Ugh.

I added a project "SyntaxTextBox" to the "Game Studio" solution, set the build order so that SyntaxTextBox was a dependancy of the project "Main Editor".  Then I added a reference of SyntaxTextBox to the Main Editor project.  I added a user control "CodeControl" and a class "EnhancedTextBox" to the SyntaxTextBox project.  I right clicked SyntaxTextBox and click Build.  Then Save All

Now, on the Main Editor project, I can add an instance of the CodeControl to the main form.  This allows me to debug my custom user control as it edit it.  I just have to make sure I always right click the UI project and click build before my changes will show up under the main form.

On CodeControl, I added a combo box, a HScroll and VScroll.  I opened the code view on CodeControl, and created a public instance WithEvents for the EnhancedTextBox.  In Public Sub New of CodeControl, I added the control to Me.Controls (after InitializeComponent), set the location, size, and anchor properties.

In EnhancedTextBox, I inherit Groupbox in Public Class EnhancedTextBox, which essentially turns my EnhancedTextBox into a groupbox (for now).

I built the project, and added the control to the main form and build the project.  As I expected, the custom control now has a combo box at the top, scroll bars on the right and bottom, and a group box filling the rest of the control.

Now on to turning the group box into a text box.  I opened the code in EnhancedTextBox, and added the methods Public Sub New, Protected Overrides Sub OnPaint, and the Public Event DrawMe.

Here's the EnhancedTextBox class now:

Public Class EnhancedTextBox
    Inherits GroupBox

    Public Sub New()
        MyBase.New()
        Me.SetStyle(ControlStyles.AllPaintingInWmPaint Or ControlStyles.UserPaint Or ControlStyles.OptimizedDoubleBuffer, True)
        Me.UpdateStyles()
    End Sub

    Public Event DrawMe(ByRef e As System.Windows.Forms.PaintEventArgs)

    Protected Overrides Sub OnPaint(ByVal e As System.Windows.Forms.PaintEventArgs)
        'don't need this, we are doing user painting
        'MyBase.OnPaint(e)
        'clear client area
        e.Graphics.FillRectangle(Brushes.White, Me.ClientRectangle)
        'draw boarder
        e.Graphics.DrawRectangle(Pens.DarkGray, Me.ClientRectangle)
        'call the draw event where text painting will occur
        RaiseEvent DrawMe(e)
    End Sub

End Class

SetStyle allows me to set flags for the control that determines how it will behave. 
AllPaintingInWmPaint effectively disables the default group box painting code
UserPaint tells the blitter that I'm going to be painting, so don't update the screen until I'm done.
OptimizedDoubleBuffer tells the blitter that I want to draw to an off screen plane, then present the whole control at once instead of drawing each item to the screen one at a time.  This gets rid of flickering.

Now in the CodeControl, we have a group box that isn't a group box any more, but will still behave as if it were one.  We need to change that.
We need to write a handler for MouseClick.  This will manually set focus on the group box so that KeyPress will function.  This method will also need to store the mouse_x, and mouse_y into a public variable, and flag that a click occurred in the group box. 
We need to write a handler for GotFocus that turns on a timer control (500 msec interval), and flags the caret on.  GotFocus will also trigger a refresh for the groupbox.
We need to write a handler for LostFocus that turns off the timer, and flags the caret off.
We need to write a handler for the timer_tick that toggles the carret on/off.
We need to write a handler for KeyPress that handles keystrokes (updating text manually, checking for keywords, setting color, etc).  I will use a select case block to determine what keys were pressed, and check for current states, such as "is text selected?  This key code replaces that text", and "are we in insert or overwrite mode?".  Some reference codes: Asc(e.KeyChar): 8 = Backspace key, 9 = Tab Key, 12 = Enter Key.  27 = Escape. 
We need to write a handler for KeyDown/KeyUp that handles Shift, Alt, Control, Arrow keys, F1 .. F12, Delete key, Page Up, Page Down, Insert (for insert/overwrite mode), and alt-menu key. 
We need to write a handler for the custom DrawMe event that actually draws the text on the screen.  If the click occurred flag is set, we need to calculate the caret line and column location while drawing the text on the screen.  This is not so obvious to do since some fonts are proportional.  We'll have to check each character as we draw it to see if the mouse clicked over it's location on screen.
We need to write a handler for MouseDown/MouseMove/MouseUp that tracks mouse movement while buttons are down to handle text selection.  I haven't worked out exactly how this will work, but I have a general idea that it will be tied with the set caret code from MouseClick.

Some of these handlers I'm working on now, and are in a basic skeleton format.  The result is I have a marginally functional text box that I can begin to do whatever I want with it.
There is a lot of code still to be written before this custom text box is ready for prime-time though.  Sounds like some straightforward programming fun to me!

Hopefully, this will give someone a basic idea for how to implement their own custom UI elements.

Expression Evaluation

Expression evaluation is one of the most difficult and complex parts of this project, but it's one of the most important.  Most data processing an any program is going to involve expressions, so it is essential that I optimize this process as much as possible.  I created 4 different flow charts approaching this problem from different angles, and think I've got a good handle on how this is going to come together. 

The first way I tried involved a large for loop that would cycle through each character in the expression, and use a state machine to separate the characters into numerals, or variables, or operators.  It used a select case to cycle through the operators, and evaluate them one at a time for left and right operands.  I'm not sure this process would work, as it doesn't handle order of operation very well.

Next idea was to pre-parse the expression before trying to evaluate it.  This would involve a large loop similar to the first idea, but no evaluation would happen in stage one; it would just parse up the terms into numbers, variables, operators, etc.  These terms would be stored in an array that the second phase would then evaluate.  It looked like the second loop would be much more compact, since it would just be a bunch of select case statements over and over.  Order of operations would not be such a problem in this model, as the evaluation routine could loop over the collection of terms, evaluating the highest precedence operators first, and working its way down through highest precedence to the lowest (parenthesis, multiplicatives, additives, relational comparisions, assignment, sequential evaluation, etc).  I liked the way this one was organized, but it would require multiple (many, many actually) loops over the terms, moving them around, combining them, and sorting them, over and over.  This would surely require a lot more processing time than I wanted to devote to this.

I thought about using dynamically compiled C to evaluate the expressions, and did some research into what would be involved in this.  I decided against it as it would require even greater processing time than the previous idea.

Finally, I settled on a single-pass method that uses recursion to evaluate expressions, selectively processing or deferring for the next operand/operator in the expression.  This method seems the most promising, and it's the one I'm going with (unless I think of a better way).

Here's the basic idea for how this works:
1) A starter function is called.  The function takes 3 parameters, the ScriptObject byref(containing variables, structs, enums, datablock, etc), the expression (as a string) byval, and a variable to hold the return value of the evaluation, byref.  The function returns an integer type enum, ErrorCode, that indicates if something went wrong.  Before it does anything, it makes a backup copy of the ScriptObject, including all Variables, structs, enums, datablock, everything.  This allows all processing to be reverted in the event of an error occuring during evaluation.  Variable contents could be left in a very inconsistent state in this case, and the only choice is to revert everything and return an error code.  This will allow easier debugging, since analyzing a total mess of jumbled data is a nightmare.

2) The starter creates a few variables to track the progress of the evaluation, and control the other functions.  Pos is the character position in the Expression where evaluation is currently at.  TermIdx is an auto-increment index assigned to each term as they are processed, for easy identification during processing.  ExpTerm is a structure to contain parsed expression terms.  ThisTerm is a var of type ExpTerm that holds the currently processing term.  LastTerm is a var of type ExpTerm that contains the previous term that was handled (or left operand for operators that take two terms).  RightTerm is a var of type ExpTerm that holds the next term to be processed (or a right operand for operators that take two terms).  For operations that need to look-ahead in the expression to make decisions (for example, declaring variables takes a ThisTerm - DataType, a RightTerm - Variable Name, and needs to look ahead to see if there are array subscripts to choose wrether to make a simple variable or an array), RightTerm2 holds the look-ahead term.  If the RightTerm2 turns out to be something unexpected, or unusable for ThisTerm, RightTerm2 is moved into NextTerm for safe keeping.

3) The Starter calls a function that parses the expression into terms, GetNextTerm.  This function takes 5 parameters, and return an ErrorCode.  Param 1: ScriptObject that contains variables, structs, enums, datablock, etc; Param 2: Expression to be parsed, Param 3: NextTerm, byref, that will be populated with the NextTerm parsed; Param 4: Pos, the position in Expression to resume parsing at; Param 5: TermIdx, the auto-increment index for the retrieved term.  This function loops through characters in the Expression, starting as Pos, ignores whitespace until the first character is found.  It checks the character for operator symbols, and if found, it identifies the operator and stores in NextTerm. 

If the first character found is alphanumeric, it builds a string Symbol until a non-alphanumeric character is reached.  It then checks Symbol to see if it is numeric: if so, it checks if the stop character was a decimal point, and continues to build the number if it was.  Once the number is fully parsed, it checks it against some constants to see what kind of numeral is it, floating point, 32-bit integer, or 64-bit interger, based on size, and decimal place existance.  It creates a Term out of the parsed info and returns.

If the Symbol is non-numeric, it assumes some sort of moniker is present.  It first checks to see if Symbol is a primitive data type: if so, it returns the Term as a DataType.  Next, it searches through the Moniker Index to find out if Symbol is an enum type, struct type, functoin call, or variable.  If enum or struct, it returns the DataType.  If function, it returns a function call Term.  If variable, it grabs the details of the variable from the VarCatalog, and returns Variable Term.

Lastly, if it still hasn't identified Symbol, it returns an UnknownMoniker Term.  This isn't an error condition, it just indicates that this term could be a new variable declaration.  Further evaluation will be required to determine if the Moniker is actually a reserved work and an error should be thrown.

4) Now that the first Term is parsed and ready to process, the Starter invokes the recursive Evaluate function.  This function takes 7 parameters, and returns ErrorCode.  Param 1: ScriptObject byref; Param 2: Expression to evaluate; Param 3: Result of the evaluation, byref; Param 4: Pos, byref; Param 5: TermIdx, byref; Param 6: ThisTerm; Param 7: LastTerm. The function is called with all available info at this point, and LastTerm is assigned Nothing, since there wasn't a last term.

5) Evaluate runs a Select Case against the type of ThisTerm.  For each possible Term type, RightTerm is populated if needed, then RightTerm2 if populated to check for precedence.  If the current term is higher priority than RightTerm2, the current Operation is evaluated and then Evaluate is called recursively; The results of this operation are LeftTerm, RightTerm2 becomes ThisTerm, and all other variables are passed on as well.  The Evaluate returns, its Result is passed on as this recursion's Result, and the function returns.

If RightTerm2 is higher precedence, then Evaluate is called before evaluating ThisTerm; LeftTerm is populated with RightTerm, and ThisTerm is populated with RightTerm2, all other vars are passed on to the recursion.  When Evaluate returns, its Result will contain the results of processing, and those are used as RightTerm.  The Operation ThisTerm is then evaluates against LeftTerm and Result, and the results are returned in Result, and the function returns.

6) After all terms have been evaluated, RightTerm2 will start populating with EmptyTerm, which is the lowest precedence priority, and all operations will complete, and if they were deferred, they will evaluate now.  Then each recursion will get a EmptyTerm when they populate RightTerm2, and eventually all Evaluate recursions will return, and execution will fall back to the Starter.

7) Starter will then analyze the ErrorCode that was returned.  If everything is ok, the returned Result will be packaged into this function's Result parameter, and execution will return to the main code execution body.  All variable assignments will be committed at this point.  If something goes wrong, and the code hasn't indicated it can handle the exception (unhandled exception), then execution is halted so I can analyze the two data blocks (the original and post-mortem versions) and the passed expression and all parsed Terms to find out what went wrong.  If the code indicates it handles exceptions, then ScriptObject will be reverted to the backup and an ErrorCode will be returned.  This allowed the script code to continue without worrying about garbled datablocks.


So, that's the plan.  I think it is a workable solution to the problem.  If anyone has any ideas or suggestions (or potential pitfalls), I'd love to hear them in the comments.

Script Engine Flow Chart

I'm close to finalize the design of the scripting engine, so I'm posting some preliminary pseudo code.

The general flow for script execution is: Load, Execute on Update, CleanUp


First, the function that loads a script from a URI (Uniform Resource Identifier)
Function LoadScript(URI As String) As Error
   Dim Script As ScriptObject = Nothing
   If IsLegalPath(URI) Then
      Script = LoadScriptFromFile(URI)
   ElseIf IsLegalURL(URI) Then
      Script = LoadScriptFromURL(URI)
   Else
      Return Error.UnrecognizedURIFormat
   End If

   If Script Is Nothing Then
      Return Error.LoadingScript
   End If

   Dim ErrCode As Error = PreParseScript(Script)
   If ErrCode <> Error.none Then
      Return ErrCode
   End If
End Function

Execute on Update is executed just before each frame is called.  Execution of scripts takes place in a single function, and the function returns when the update is complete and the graphics engine can begin rendering objects to the screen.  The first time this function is called for a script, it begins execution of the script at the top of the main() function.  On subsequent calls, it resumes execution where it left off before.

Function ExecuteScript(Script As ScriptObject) As Error
   Dim ErrCode As Error
   If Script.PC = -1 Then
      ErrCode = CallScriptFunction(Script, "main")
      If ErrCode <> Error.none Then
         Return ErrCode
      End If
   End If

   ErrCode = RunCode(Script)
   If ErrCode <> Error.none
      Return ErrCode
   End If

   Return Error.none
End If

There are structure and enums that hold information about identified functions, variables, structs, enums, and the locations in the script code they are found.  These structures hold information such as data type expected, memory contents of variables, and a tree-like structure for tracking structs and arrays.  The idea is that the PreParse function can identify all this info before execution begins, thereby speeding execution of the code. 

The function RunCode is where all the action is at.  It gets symbols from the script one at a time, and interprets what they mean.  The whole function is inside a big loop, and repeatedly gets symbols and interprets them, until a command is encountered that flags the script's readiness to update the display screen.  Once the script flags that it is done running for this frame, the function returns an Error code (Error.none for success), and the graphics engine begins to update the world with data processed and returned by the script.

All script state information about nesting, function calls, variable contents and scope, and instruction pointer, are all contained in the ScriptObject.  This allows the script to trigger an update at just about any point and safely expect execution to resume exactly as it was upon next update cycle.

When the script is finished up (the game is over) the main() function ends, and memory cleanup is initiated.  After this cleanup process, the game engine is notified to shut down, and objects are disposed, such as sounds, images, models, network connections, etc.  Then the engine exist.

There is one big part missing from this script engine so far: expression evaluation.  This topic is so long and complex, it deserves its own separate post.  Expression evaluation is likely to be the most complex part of the entire game engine.  I envision a recursive token parsing evaluating system, and as soon as I figure out how to implement this, there will be a lengthy post about it.

More updates to come as design continues.

Scripting Plans

I've taken quite a few formal programing classes over the years, but none of them have even mentioned the concept of scripting, much less emulating one language with another.  I find myself asking the question "Where do I start?"  Well, it seems reasonable to search the Internet and see if anyone else has tried what I'm about to do.  As it turns out, if anyone is, they aren't advertising it to the world at large.  No big surprise there though. 

Well, I came up with a plan, a roadmap if you will, for how I'm going to pull this all together to make it work.  It starts with a formal definition of exactly what language I'm implementing.  This isn't just a matter of picking a language that already exists; I have to define what commands are going to be available, what their syntax is, and how they are to function, exactly.  I've never thought about it before, but programing languages are complex, and very difficult to describe in precise terms.

For this project, I've chosen to implement my own version of C.  I'm going to try to make it as close to ANSI C as possible, but there are going to be differences, for sure.  I've typed up a document that formally defines every data type, command, structure, and the syntaxes for all.  It helps that I'm comfortable with C, so I didn't have to do very much research in this task.  Here's a brief overview of my work so far.

Primitive Data Types:
void - indicates a function type returns no data.
int - 32-bit signed integer.
long - 64-bit signed integer.
float - 64-bit IEEE double-precision floating point number.
char - 2 byte Unicode character.
cstr - A string, similar to a c-style string, but 2 bytes per character (for Unicode support), and terminated with a two 0 bytes ( \0\0).

User-Defined Data Types:
enum - an enumeration of values.  Internally, this is the same as an int, but you can assign monikers to represent numbers.
struct - a collection of other data types that can be stored in a single variable, or used as an array.
Arrays - a collection of the same data type stored in a single variable, using subscripts to access individual elements of the array.  Can be made of any primitive or user defined data type.

Declaring variables:
primitive data types: datatype moniker [ = initial value];
enums: enum moniker { [member1 [ = value ] [, member2 [ = value] ...]] }[moniker];
structs: struct moniker { [type1 moniker1; [type2 moniker2;]] } [moniker];
arrays: datatype moniker[arraybound1][arraybound2]...;

Declaring functions:
prototype function: datatype moniker([datatype [,datatype...]]);
define function: datatype moniker ([datatype param1 [, datatype param2...]]) { statements; };

Now for some command statements.  These fall into a few categories.

Looping:
for ([init_expression]; [condition_expression]; [loop_expression]) {statements;}
do {statements;} while (condition);
while (condition) {statements;}

Condition Execution
if (condition) {statements;} [else {statements}]
switch (expression) { case const_value1 : [statements;] [break]; [ case const_value2 : [statements;] [break;] ] ... [default : [statements;] [break;]] }


For those of you who are not as familiar with c-style syntax, here's a few examples to help clarify what all these definitions mean.

Variable declaration and assignment.
int a = 4;        /* creates an int var, assigns value 4 */
int b;     /* creates int var b, default value is 0 */
b = a;    /* assigns var b the value in var a (4) */
a = 5 + b;   /* assigns var a the value of 5 plus the value of var b (4), so 9 altogether */

Array definition and handling.
int list[2];   /* creates an array variable list, with two elements */
list[0] = 2;  /* assigns the first element, 0, the value 2 */
list[1] = 5;  /* assigns the second element, 1, the value 5 */

Enum definition and handling.
enum DaysofWeek { Sunday, Monday, Tuesday, Wednesday, Thursday, Friday, Saturday };
DaysofWeek Today = DaysofWeek.Tuesday;
DaysofWeek Tomorrow = Today + 1;   /* assigns the value of Today (2) + 1, to var, will = Wednesday (3) */

Struct definition and handling.
struct coordinates {
    int X;
    int Y;
};   /* creates a struct called coordinates, that contains two data fields, an int called X, and an int called Y */
coordinates Location;   /* defines a variable called Location of struct type coordinates.  You C guys will notice the lack of the struct keyword in this definition.  I'm leaving it out of my specification because it's not necessary. */
Location.X = 3;   /*assigns the X component of the Location coordinates structure the value 3 */
Location.Y = 2;   /*assigns the Y component of the Location coordinates structure the value 2 */


Looping example:
For loops are generally used when you need to run a block of code a fixed number of times.  Before the loop is started, the init_condition is evaluated.  This is generally used to create local variables for a loop counter.  Then the condition_expression is evaluated to see if the statements get execution.  If the condition_expression evaluates to non-zero, then statements are executed, else the block of statements is skipped, and the for ends. After execution of the statements, the loop_expression is evaluated, then the process starts back over at testing the condition_expression.  All parameters are optional, and if condition_expression is omitted it is assumed to be non-zero, and an infinite loop is created.  A for loop can be exited early with the break; statement, or prematurely restarted (just like it had reached the end of statements) with a continue; command.
int total = 0;
for (int count = 1; count < 5; count ++) {
    total += count;
}     /* total will be equal to 10 when this finishes */

Do loops are generally used when you don't know how many times a block of code needs to run, but you know it needs to run at least once through.  The loop condition is tested at the end of each iteration.  Do loops can be exited with a break; command.
int a  = 1;
int b = 32;
do {
    a +=1;
} while (a < b);

While loops are generally used when you dont' know how many times a block of code needs to run, and in fact, it might not run a single time.  The loop condition is tested before entering each iteration.  While loops can be exited with a break; command.
int a = 3
while (a < 100) {
   a +=1;
}

Conditional execution example.
If statements are used to test a condition, and execute statements if non-zero, and optionally execute statements if zero. 
int a = 3;
int b = 2;
if (a > b) {
    /* a is bigger than b */
} else {
    /* a is smaller than or equal to b */
}

Switch statements are used when testing an expression against multiple possible values.  Switch is preferable to a series of if statements, but can only be used to test against constant values.  Switch evaluates the expression, then begin search each case clause.  When a case is matches, execution begins following the : until a break; statement is reached.  break; sends execution to the end of the statement block.  An option default statement starts execution after the : if no previous case values matches the expression.
int a = 3;
switch (a) {
    case 1 :
        /* a = 1 */
        break;
  case 2 :
        /* a = 2 */
        break;
   default :
        /* a did not equal 1 or 2 */
        break;
}

Well, that's all the examples I have for now.  If anyone has any thoughts or suggestions about this, please feel free to post them in the comments below.

Next up, how to read this programmatically.

Scripting technical challenges

As I work on this game engine, slowly but surely I'm learning new programing strategies.  Here's something I've learned in the last 24 hours: It is one thing to write a program; it is entirely different to write a program to emulation another programing language.

Let me further define the situation here.  When you write a program, a compiler turns your code into machine language (or some sort of pseudo code anyway).  When your program is running, there are a series of transistors that physically turn on and off electricity to components of your CPU, using the electrical impulses that comprise your program's binary digits machine code to direct that activity.  The part of your CPU that gets activated begins to carry out activities by turning further transistors on and off , all in response to the pattern of electrical impulses you programed for it.  One part of your CPU causes a word of data to be read from  memory and placed into a register.  Another part might take that word of memory and perform some sort of math function on it.  Other parts of your CPU send signals out to your video card, and those signals further instruct that video card to read memory.  Those signals cause the video card to generate a new signal to your monitor, allowing you to read this excessively verbose test.

The point is, all the processing happens at a physical level in the CPU; physical transistors being turned on and off by our inputs and the program's instructions.  This all happens extremely quickly, and all in the background, so it is easy to forget what is happening at the lowest levels of our machines.

When your program supports scripting, you have to read in some sort of file, and the patterns of characters tells your program how to carry out certain pre-programmed tasks.  If you want your script host to be able to pop up a message box for the user, you have to define a pattern of characters that your program can recognise, that direct it to do so.  Your program spends a great deal of time analyzing the characters in this script file, and your program has to be careful how much time it devotes to doing this.  If your program is a game engine, you will already be spending a great deal of time telling the video card where and how to display any number of graphics on the monitor.  You don't want to take away from that time, or the game will start running slower. 

To compare how quickly the physical CPU is at running a program to a piece of software analyzing one and emulating it on the fly, is very difficult.  I would estimate the emulation to be many hundreds of times slower; perhaps even thousands of time slower.

Imagine you want your game to run at 60 frames per second.  That means your program has 1/60th of second (or about 0.0166 seconds) to check the keyboard, mouse and controllers, figure out what that input means to the game, and take action on it, update all of the physics for all the objects in the game, then finally tell the video card where and how everything is supposed to look on the monitor.  Online games also have to deal with incoming data from the server, updating the world objects accordingly, and prepare and send packets of data back to the server to let it know what input the game got from the user. 

Computers are fast, but this can be a LOT of work to do in such a short amount of time.  And, the game isn't the only thing the computer is doing at that moment in time.  Your program doesn't get to run continuously when it is running.  Instead, windows assigns "time slots" for your program to run in, called a quantum.  A quantum is a unit-less measure of CPU time.  For most desktop computers, the 3 quantum is going to equal about 10 milliseconds on a single processor system, or 15 milliseconds on a multi processor system.  How many quantum your program gets for execution each time it is scheduled depends on a number of very complex factors, but can be anywhere from 3 to 12 quantum. 

Once you start to realize that your program doesn't necessarily get time slices that line up with your desired 1/60th of a second, you notice there's a problem here.  If your program spends its entire quantum of time running a script, you will miss your window of opportunity to display an updated frame.  Too much of that, and your program will run very choppy.  Your monitor's vertical refresh pulse can't wait around for your game to catch up, of the display will start to flicker, a subjectively even worse condition.

So, I'm going to work on these challenges as I design the script engine, and if I come up with anything clever, I'll be sure to post them.

C-Scripting developments

After spending the entire night working on this, I've come up with what I hope is a workable plan.  Instead of making a game engine and tagging on a script language to control it, I want to rework the entire engine to focus primarily on the script language, then adding functionality to the language by way of APIs.  This would reshape the entire project to wrap all the game engine elements around the scripting language; I think this may be the best way to build the game engine.

Anyway, I'm working on a language syntax definition document, and as soon as it's ready, I'll post it here for comments and suggestions.  I will also be putting together an API so the script language can control things in the game engine. 

As it is now, XNA gives us two important functions, Update, and Draw.  Update is where all the game logic is supposed to take place, and Draw is where all the game rendering happens.  Initially, I wanted to run the scripts in the Update function, and have a fully built and updated world object that the Draw function can quickly render to the screen.  However, I recognize that there are two completely different paradigms here I am trying to balance: procedural, and event-driven programming.

In a procedural design, there would be one script running at a time.  It would be responsible for loading, controlling, displaying/using, and disposing of all game resources.  This means that a game script would basically create a bunch of objects and loop through them every frame update processing game logic and what-not.  This is the easiest to implement I think.  Scripting would happen exclusively in the Update function, and any rendering APIs would build a world object that the Draw function would then render.  After Draw finishes, Update would be called again, and the script would pick where it left off last frame.  The scripting language would need a command to indicate when it is ready to sync, thereby pausing execution so the prepared game scene can be presented to the user.

In an event-driven design, I can imagine a "master" script creating game objects, and attaching scripts to each object.  As the game runs, users click on objects and type on the keyboard, objects collide, or one object sends messages to other objects. These actions trigger messages to be dispatched to each object.  Objects can bind certain message types to functions, thereby creating event handlers.  Think of windows programming, where you click on a button and the program executes code in response to that.  The same sort of thing is going to be happening in the game.  One strength of this is that independent agents can be running from their own namespaces, meaning that a player can be controlling their character, while an enemy stalks that player down or plans a strategy.  These sections of code can inherently be run in parallel, meaning the game can take advantage of modern multi-core processors.  Unfortunately, there are two big downsides here. 1) designing a game to run in such an environment is extremely difficult and complex.  When bugs creep in, finding them and fixing them can be very challenging, and as the complexity of the game goes up, you begin to see emergent behavior in the independent agents.  The goal of programming this way is to create the emergent behavior to be exactly what you want, and have a very good game.  2) designing a game engine to work this way seems to be extremely complex.  Instead of having a script interpreter, and a few classes for handling custom user data, now we have a full inter-process messaging system and process queue.  It's kind of like developing a mini-operating system instead of just a single program.

So for now, I'm going with the procedural programming paradigm, but I'm always open to comments.  If you have an opinion one way or the other, feel free to share them and discuss below!