Overview

CodeWorker is a scripting language distributed under the GNU Lesser General Public License and devoted to manipulate many aspects of generative programming as easy and intuitive as possible. Generative programming is a software engineering approach for producing reusable, tailor-made, evolvable and reliable IT systems with a high level of automation.

The scripting language adapts its syntax to the subject it has to handle: - an extended-BNF syntax (declarative part of the language) for recognizing the format of the specifications to parse, - a procedural language for manipulating easily parse trees (the only structured type admitted by 'CodeWorker'), strings, files and directories, - a JSP-like syntax (imperative part of the language), which facilitates the writing of template-based code generation.

Thanks to this syntax adaptation, the scripting language is able to easily: - acquire any kind of specification of the IT system to produce (often XML but not necessary), - generate source code in a classical way (as Rational ROSE), managing protected areas of text that accept hand-typed code, - expand a source file like the class-wizard of Visual C++ (generated text is inserted at specified markups), - translate from a format to another (LaTeX to HTML, XSL to CodeWorker, ... no limit), - transform a source file (to instrument a source file with profiling features, ...).

These tasks are executed in a straightforward process, with no binding to an external programming language and with no translation of requirements specification.

CodeWorker provides two methods for performing a parsing:

• the reading of tokens is procedural,
• the BNF description is declarative, and conforms to a kind of BNF (the Backus-Naur Formalism represents a grammar in a particular syntax) extended with regular expressions,

We suggest to use the file extension ".cwp" for extended-BNF parse scripts.

Given a specification provided in any kind of format, CodeWorker will generate source code or text as required in template-based scripts.

The source code generation can use three modes: generation, expansion or translation.

• generation mode is used to let the script produce the most part of the output file, processing a kind of template-based generation as it exists for a JSP or PHP script. Only some areas called protected areas in the vocabulary of CodeWorker are preserved in the file. This philosophy has been adopted by some modeling tools that generate a skinny skeleton copiously interspersed with areas intended to the developer.
• expansion mode is used when the file is mainly written by hand, but small portions need to be generated. The points where to insert the code are called markups in the vocabulary of the scripting language. The Class Wizard of Visual C++ changes source code following this principle.
• translation mode is used when both parsing and source code generation are required to process a file. It arrives for processing:
  • a source-to-source translation: a file must be rewritten in a different syntax. For example, a LaTeX file might have to be translated in HTML.
  • a program transformation: a source file has to change for optimizing, refactoring, instrumenting or rewriting some portions.
    For example, a script could add a trace at the beginning of each function body of a JAVA or C++ source code. To do that, parsing will serve to discover function bodies, and source code generation will insert the C++ or JAVA code that implements the trace.

We suggest to use the file extension ".cwt" for template-based scripts.

A formal representation describes all functions and procedures that CodeWorker provides, with their prototype and a short explanation and an example and the list of all-similar functions and procedures. This formal representation is used to generate source codes of CodeWorker that handle parsing and C++ mapping and execution of each function and procedure of the scripting language. This formal representation that conforms to what CodeWorker expects in terms of function/procedure prototypes, is reused to generate the LaTeX part of the reference manual that describes each of them. Examples are executed while generating the documentation to be sure they are correct, and to report an up to date output.

The chapter getting started is partially generated too, and the guarantee is given that every script runs successfully and that every example file has the last annotations. To warrant that, scripts are executed while generating the documentation, and example/script files contain some formatted comments just before lines to annotate. While including them into the chapter, their content is numerated line by line, and notes are extracted. Notes are written just after the content, and refer to the line they explain.

The documentation is written in LaTeX. The great advantage of LaTeX is that it offers a powerful text processing and that it is easy to manipulate for source code generation (text format instead of binary, and it accepts comments). Markups are inserted into the documentation at the points where generated text must be included. A markup is a special comment that CodeWorker recognizes. This mode of code generation is an illustration of what is called expansion mode before.

This chapter is intended to help you to discover the scripting language and how it may serve your software development process.

CodeWorker is delivered with:

• an executable called CodeWorker, which runs into a shell and that requires options on a command line,
• a library called CodeWorker.lib, which may be linked to C++ applications for extending them with parsing and source code generation feature,
• some C++ headers that allow exploiting the library and that are available into the "include" directory,

Binaries are available into the "bin" directory.

The scripting language adapts its syntax to the nature of the tasks to handle:

• Acquiring the specifications of what to generate requires to be able to recognize format, either invented for answering as fine as possible to the particularities of the any kind of project or existing on the market and available on the script repository (see http://www.codeworker.org/ScriptsRepository.html, in constant improvement). A declarative language processes the scan of the format, following an extended BNF syntax ; it accepts the intrusion of procedural instruction to populate the parse tree.
• Manipulating internal data easily and executing instructions with an expressiveness similar to a classical system programming language. The procedural part of the language enables to take them in charge.
• Generating, expanding or tranforming text. The imperative part of the language offers a template-based syntax that accepts instructions to navigate into the parse tree and to take advantage of facilities brought by a programming language.

Example:

CodeWorker allows saving time to implement source code, if it disposes of a detailed design. Let start with a tiny modeling language that only understands object types and that we create just for this example:

      // file "GettingStarted/Tiny.tml":
      1 class A {
      2 }
      3
      4 class B : A {
      5 }
      6
      7 class C {
      8     B[] b
      9 }
      10
      11 class D {
      12     A a
      13     C[] c
      14 }

The role of the parsing is to populate the parse tree. Let suppose that, for each class, we need of the following attributes:

• name: the name of the class, C for example,
• parent: the name of the parent if exists, A for class B for instance,
• listOfAttributes: an array that contains the description of encapsulated attributes, a and c into class D for instance,

• name: the name of the attribute, a into class D for instance,
• class: the name of the class it belongs to, A for attribute a for instance,
• isArray: true if the attribute is an array like c for example,

To discover the parse tree, we'll first populate it by hand. To do that, let run CodeWorker in console mode:

CodeWorker -console

Type the following line into the console, and be careful not to forget the final semi colon:

insert listOfClasses["A"].name = "A";
traceObject(project);

The insert keyword is used to create new branches into the parse tree. The root is named project, but hasn't to be specified, and a sub-node (or attribute) listOfClasses has been added. This sub-node is quite special: it has to contain an array of nodes that describe classes. Items are indexed by a string and are stored into their entrance order; so, the node that takes in charge of describing the class A is accessed via listOfClasses["A"]. The string "A" is assigned to the attribute listOfClasses["A"].name.

The procedure traceObject(project) shows us the first-level content of the root: the attribute listOfClasses and all its entries (only "A" for the moment). Let populate the tree with the description of the class B:

set listOfClasses["B"].name = "B";

The set keyword is used to assign a value to an existing branch of the parse tree. If this branch doesn't exist yet, a warning notices you that perhaps you have done a spelling mistake, to avoid inserting new bad nodes. But the node is inserted despite of the warning. As the language isn't typed, it allows avoiding some troubles. Let's continue:

ref listOfClasses["B"].parent = listOfClasses["A"];
traceLine(listOfClasses["B"].parent.name);

The node listOfClasses["B"].parent refers to the node listOfClasses["A"], so listOfClasses["B"].parent.name is similar to listOfClasses["A"].name. Let start filling in the tree for class C:

insert listOfClasses["C"].name = "C";
pushItem listOfClasses["C"].listOfAttributes;
local myAttribute;
ref myAttribute = listOfClasses["C"].listOfAttributes#back;

The pushItem assignment command is another way to add a new node into an array, where the item is indexed by the position of the node, starting at 0. The local keyword allows declaring a variable on the stack. This variable is also a parse tree, but not attached to the main parse tree project. For more commodities, this variable will refer to the last element of the attribute's list: myAttribute is shorter to type than listOfClasses["C"].listOfAttributes#back. Notice that the last element of an array is accessed via '#back'. Let complete the attribute b of class C:

insert myAttribute.name = "b";
ref myAttribute.class = listOfClasses["B"];
insert myAttribute.isArray = true;

The keyword true is a predefined constant string that is worth "true". The keyword false also exists and is worth an empty string.

Exercise:

Populate the parse tree with the description of class D.

We have to improve the precedent script, called now "Tiny-BNFparsing.cwp", for building the parse tree that represents the pertinent data of the design:

      // file "GettingStarted/Tiny-BNFparsing.cwp":
      1 TinyBNF ::= #ignore(JAVA) [classDeclaration]* #empty
      2     => { traceLine("this file has been parsed successfully"); };
      3 classDeclaration ::=
      4     IDENT:"class"
      5     IDENT:sName
      6     => insert project.listOfClasses[sName].name = sName;
      7     [
      8         ':'
      9         IDENT:sParent
      10         => {
      11             if !findElement(sParent, project.listOfClasses)
      12                 error("class '" + sParent + "' should have been declared before");
      13             ref project.listOfClasses[sName].parent = project.listOfClasses[sParent];
      14         }
      15     ]?
      16     classBody(project.listOfClasses[sName]);
      17 classBody(myClass : node) ::=
      18     '{' [attributeDeclaration(myClass)]* '}';
      19 attributeDeclaration(myClass : node) ::=
      20     IDENT
      21     ['[' ']']?
      22     IDENT;
      23 IDENT ::= #!ignore ['a'..'z'|'A'..'Z']+;

line 5: the name of the class is put into the local variable sName. Note that the first time a variable is encountered after a token, it is declared as local automatically.
line 6: we populate the parse tree as we have proceeded manually,
line 9: the name of the parent class is put into the local variable sParent,
line 11: the parent class must have been declared before: the item is searched into the list of classes,
line 13: we populate the parse tree as we have proceeded manually,
line 16: clauses may accept parameters; here, the current class is passed to classBody that will populate it with attributes,
line 17: the clause classBody expects a parameter as a node; a parameter may be passed as value or node or reference,
line 19: little exercise: complete the clause attributeDeclaration that takes in charge of parsing an attribute of the class given to the argument myClass,
line 20: remember that you must parse the class name of the association here (attribute myClass.listOfAttributes#back.class refers to the associated class),
line 21: remember that you must parse the multiplicity of the association here (attribute myClass.listOfAttributes#back.isArray is worth true if '[]' is present),
line 22: remember that you must parse the name of the association here (to put into attribute myClass.listOfAttributes#back.name),
Exercise:

Complete the precedent clause attributeDeclaration to populate an attribute. You'll find the solution into file "Scripts/Tutorial/GettingStarted/Tiny-BNFparsing1.cwp".

Solution:

      // file "GettingStarted/Tiny-BNFparsing1.cwp":
      1 classBody(myClass : node) ::=
      2     '{' [attributeDeclaration(myClass)]* '}';
      3 attributeDeclaration(myClass : node) ::=
      4     IDENT:sClass
      5     => local myAttribute;
      6     => {
      7         pushItem myClass.listOfAttributes;
      8         ref myAttribute = myClass.listOfAttributes#back;
      9         if !findElement(sClass, project.listOfClasses)
      10             error("class '" + sClass + "' should have been declared before");
      11         ref myAttribute.class = project.listOfClasses[sClass];
      12     }
      13     ['[' ']' => insert myAttribute.isArray = true;]?
      14     IDENT:sName => {insert myAttribute.name = sName;};
      15
      16 IDENT ::= #!ignore ['a'..'z'|'A'..'Z']+;

line 4: the name of the class for the association is assigned to the local variable sName,
line 5: we'll need a local variable to point to the attribute's node for commodity,
line 7: the local variable myAttribute hasn't been declared here, because it disappears at the end of the scope (the trailing brace); a new node is added to the list of attributes,
line 8: the local variable myAttribute points to the last item of the list,
line 9: the class specifier of the association must have been declared,
line 11: we populate the parse tree as done by hand,
line 13: this attribute isArray is added only if the type of the association is an array,
line 14: we complete the attribute description by assigning its name,
Type the following line into the console to parse the design "Tiny.tml":


parseAsBNF("Scripts/Tutorial/GettingStarted/Tiny-BNFparsing1.cwp", project, 
        "Scripts/Tutorial/GettingStarted/Tiny.tml");


this file has been parsed successfully

Now, we'll implement a little function that displays the content of our parse tree. We stop using the console here, and we'll implement the call to the parsing and the function into a leader script. This script will be called at the command line, as seen further.

We suggest to use the file extension ".cws" for non-template and non-BNF scripts.

CodeWorker command line to execute:
-script Scripts/Tutorial/GettingStarted/Tiny-leaderScript0.cws

      // file "GettingStarted/Tiny-leaderScript0.cws":
      1 parseAsBNF("Tiny-BNFparsing1.cwp", project, "Scripts/Tutorial/GettingStarted/Tiny.tml");
      2
      3
      4 function displayParsingTree() {
      5     foreach i in project.listOfClasses {
      6         traceLine("class '" + i.name + "'");
      7         if existVariable(i.parent)
      8             traceLine("\tparent = '" + i.parent.name + "'");
      9         foreach j in i.listOfAttributes {
      10             traceLine("\tattribute '" + j.name + "'");
      11             traceLine("\t\tclass = '" + j.class.name + "'");
      12             if existVariable(j.isArray)
      13                 traceLine("\t\tarray = '" + j.isArray + "'");
      14         }
      15     }
      16 }
      17
      18 displayParsingTree();

line 4: a user-defined function without parameters,
line 5: the foreach statement iterates all items of an array; here, all classes are explored,
line 7: check whether the attribute parent exists or not,
line 9: all attributes of the current class i are iterated,
line 12: perhaps the association is multiple,
line 18: a call to the user-defined function,

Output:

this file has been parsed successfully
class 'A'
class 'B'
    parent = 'A'
class 'C'
    attribute 'b'
        class = 'B'
        array = 'true'
class 'D'
    attribute 'a'
        class = 'A'
    attribute 'c'
        class = 'C'
        array = 'true'

The source code generation exploits the parse tree to generate any kind of output files: HTML, SQL, C++, ...

A pattern script is written in the scripting language of CodeWorker, extended to be able to fuse the text to put into the output file and the instructions to interpret. It enables to process a {template-based} generation. Such a script looks like a JSP template: the script is embedded between tags '<%' and '%>' or '@'.

We'll start by generating a short JAVA class for each class of the design. It translates the attributes in JAVA and it generates their accessors:

      // file "Scripts/Tutorial/GettingStarted/Tiny-JAVA.cwt":
      1 package tiny;
      2
      3 public class @this.name@ @
      4 if existVariable(this.parent) {
      5     @ extends @this.parent.name@ @
      6 }
      7 @{
      8     // attributes:
      9 @
      10 function getJAVAType(myAttribute : node) {
      11     local sType = myAttribute.class.name;
      12     if myAttribute.isArray {
      13         set sType = "java.util.ArrayList/*<" + sType + ">*/";
      14     }
      15     return sType;
      16 }
      17
      18 foreach i in this.listOfAttributes {
      19     @ private @getJAVAType(i)@ _@i.name@ = null;
      20 @
      21 }
      22 @
      23     //constructor:
      24     public @this.name@() {
      25     }
      26
      27     // accessors:
      28 @
      29 foreach i in this.listOfAttributes {
      30     @ public @getJAVAType(i)@ get@toUpperString(i.name)@() { return _@i.name@; }
      31     public void set@toUpperString(i.name)@(@getJAVAType(i)@ @i.name@) { _@i.name@ = @i.name@; }
      32 @
      33 }
      34 setProtectedArea("Methods");
      35 @}

We'll learn about another mode of generation: expanding a file. Let suppose that you want to inlay generated code into an existing file. The way to do it is first to insert a special comment at the expected place. This comment begins with ##markup## and is followed by a sequence of characters written between double quotes and called the markup key.

Here is a little HTML file that is going to be expanded:

      // file "Scripts/Tutorial/GettingStarted/Tiny.html":
      <HTML>
          <HEAD>
          </HEAD>
          <BODY>
      <!--##markup##"classes"-->
          </BODY>
      </HTML>

The markup key is called "classes" and is put into the file like it: <!- -##markup##"classes"- ->.

Now, we'll implement a short script that is intended to populate the markup area with all classes of the design, displayed into tables:

      // file "Scripts/Tutorial/GettingStarted/Tiny-HTML.cwt":
      1 @
      2 if getMarkupKey() == "classes" {
      3     foreach i in project.listOfClasses {
      4         @ <TABLE>
      5             <TR>
      6                 <TD colspan=3><B>@i.name@</B></TD>
      7             </TR>
      8             <TR>
      9                 <TD><EM>Attribute</EM></TD><TD><EM>Type</EM></TD> <TD><EM>Description</EM></TD>
      10             </TR>
      11 @
      12         foreach j in i.listOfAttributes {
      13             @ <TR>
      14                 <TD><I>@j.name@</I></TD><TD><CODE>@
      15             @@j.class.name@@
      16             if j.isArray {
      17                 @[]@
      18             }
      19             @</CODE></TD><TD>@
      20             setProtectedArea(i.name + "::" + j.name);
      21             @</TD>
      22             </TR>
      23 @
      24         }
      25         @ </TABLE>
      26 @
      27     }
      28 }

line 2: the function getMarkupKey() returns the current expanding markup that is handled,
line 3: all classes will be presented sequentially into tables of 3 columns, whose title is the name of the class, and rows are populated with attributes,
line 12: the name, Type and Description of all attributes of the class are presented into the table,
line 15: the type is expressed in the syntax of our tiny modeling language,
line 20: the description of an attribute must be filled by the user into a protected area, so as to preserve it from an expansion to another,
The leader script has to take into account the expansion of the HTML file:

CodeWorker command line to execute:
-script Scripts/Tutorial/GettingStarted/Tiny-leaderScript2.cws

      // file "Scripts/Tutorial/GettingStarted/Tiny-leaderScript2.cws":
      1 parseAsBNF("Scripts/Tutorial/GettingStarted/Tiny-BNFparsing1.cwp", project, "Scripts/Tutorial/GettingStarted/Tiny.tml");
      2
      3 foreach i in project.listOfClasses {
      4     generate("Scripts/Tutorial/GettingStarted/Tiny-JAVA.cwt", i, "Scripts/Tutorial/GettingStarted/tiny/" + i.name + ".java");
      5 }
      6
      7 traceLine("expanding file 'Tiny0.html'...");
      8 setCommentBegin("<!--");
      9 setCommentEnd("-->");
      10 expand("Scripts/Tutorial/GettingStarted/Tiny-HTML.cwt", project, "Scripts/Tutorial/GettingStarted/Tiny0.html");
      11 //normal;

line 8: to expand a file, the interpreter has to know the format of comments used for declaring the markups. If the format isn't correct, the file will not be expanded.
line 10: be careful to call the procedure expand() and not to confuse with generate()! Remember that a classic generation rewrites all according to the directives of the pattern script and preserves protected areas, but doesn't recognize markup keys.

Output:

this file has been parsed successfully
expanding file 'Tiny0.html'...

It hasn't a great interest to present here the content of the HTML once it has been expanded, but you can display it (file "Scripts/Tutorial/GettingStarted/Tiny0.html") into your browser. You'll notice into the source code that the expanded text is put between tags <!- -##begin##"classes"- -> and <!- -##end##"classes"- ->. Don't type text into this tagged part, except into protected areas, because the next expansion will destroy the tagged part.

For discovering more about CodeWorker through a more complex example, please read the next chapter. You'll learn how to do translations from a format to another, and to use template functions or BNF clauses (very efficient for readability and extension!), and a lot of various things. But it is recommended to practice a little before.

The first time, we recommend to read the precedent chapter, more approachable, before reading this one.

Let imagine that we dispose of a design expressed in a simple modeling language, like it:

      // file "GettingStarted/SolarSystem0.sml":
      1 class Planet {
      2     double diameter;
      3     double getDistanceToSun(int day, int month, int year);
      4 }
      5
      6 class Earth : Planet {
      7     string[] countryNames;
      8 }
      9
      10 class SolarSystem {
      11     aggregate Planet[] planets;
      12 }

The parsing scripts that read tokens are the oldest way to parse into CodeWorker and are the fastest mode too. But it doesn't offer the same flexibility as BNF scripts, which are syntax-oriented.

Below is an example of what a script that reads tokens looks like:

      // file "GettingStarted/SimpleML-token-reading.cws":
      1 declare function readType();
      2
      3 while skipEmptyCpp() {
      4     if !readIfEqualToIdentifier("class") error("'class' expected");
      5     skipEmptyCpp();
      6     local sClassName = readIdentifier();
      7     if !sClassName error("class name expected");
      8     skipEmptyCpp();
      9     if readIfEqualTo(":") {
      10         skipEmptyCpp();
      11         local sParentName = readIdentifier();
      12         if !sParentName error("parent name expected for class '" + sClassName + "'");
      13         skipEmptyCpp();
      14     }
      15     if !readIfEqualTo("{") error("'{' expected");
      16     skipEmptyCpp();
      17     while !readIfEqualTo("}") {
      18         skipEmptyCpp();
      19         readType();
      20         skipEmptyCpp();
      21         local sMemberName = readIdentifier();
      22         if !sMemberName error("attribute or method name expected");
      23         skipEmptyCpp();
      24         if readIfEqualTo("(") {
      25             skipEmptyCpp();
      26             if !readIfEqualTo(")") {
      27                 do {
      28                     skipEmptyCpp();
      29                     local iPosition = getInputLocation();
      30                     local sMode = readIdentifier();
      31                     if !sMode error("parameter type or mode expected");
      32                     if (sMode != "in") && (sMode != "out") && (sMode != "inout") {
      33                         setInputLocation(iPosition);
      34                         set sMode = "";
      35                     }
      36                     skipEmptyCpp();
      37                     readType();
      38                     skipEmptyCpp();
      39                     local sParameterName = readIdentifier();
      40                     if !sParameterName error("parameter name expected");
      41                     skipEmptyCpp();
      42                 } while readIfEqualTo(",");
      43                 if !readIfEqualTo(")") error("')' expected");
      44             }
      45             skipEmptyCpp();
      46         }
      47         if !readIfEqualTo(";") {
      48             error("';' expected to close an attribute, instead of '" + readChar() + "'");
      49         }
      50         skipEmptyCpp();
      51     }
      52 }
      53 traceLine("the file has been read successfully");
      54
      55 function readType() {
      56     local sType = readIdentifier();
      57     if !sType error("type modifier or name expected, instead of '" + readChar() + "'");
      58     if sType == "aggregate" {
      59         skipEmptyCpp();
      60         sType = readIdentifier();
      61         if !sType error("aggregated class name expected");
      62     }
      63     skipEmptyCpp();
      64     if readIfEqualTo("[") {
      65         skipEmptyCpp();
      66         if !readIfEqualTo("]") error("']' expected to close an array declaration");
      67     }
      68 }

line 1: forward declaration of method readType(), so as to start explanations about how to implement BNF clause world ::= [class_declaration]*,
line 3: do a loop while the end of file hasn't been reached, skipping blanks and C++ comments: skipEmptyCpp() returns false only if an error occurs while reading the stream or the file has completed,
line 4: waiting for token "class" as an identifier (doesn't accept "class" as the beginning of another identifier, such as "classes"). If not found, an error occurs. This token announces a class declaration.
line 5: a disadvantage of writing a procedure-driven reading/parsing: don't forget to skip explicitly blanks and comments by yourself,
line 6: populates a local variable with an identifier token that represents the name of the class
line 7: if an identifier token hasn't been found (token is empty), an error is thrown,
line 9: if the file location points to ":", announcing the inheritance, function readIfEqualTo(":") returns true, and the location moves after the matched expression. If it fails, the file location remains the same.
line 15: body of the class declaration expected
line 17: while inside the class body, reading of attribute and method members,
line 19: we don't conform exactly to the BNF: beginning of method and attribute declaration is factorized,
line 21: name of the attribute or method member,
line 24: not any more ambiguity : it starts by a parenthesis when the members is a method,
line 27: the method expects at least one parameter,
line 29: we keep the current file position, to be able to come back if the next token isn't an access mode ("in", "out" or "inout"),
line 33: we were reading a basic type, instead of a parameter access mode: we come back to the beginning of this token and the mode is set as empty (no mode). Of course, it is possible not to waste time like this, and to optimize function readType() by passing the token as a parameter. But here is the occasion of discovering how to handle the file position.
line 37: type of the current parameter is expected,
line 39: name of the current parameter is expected,
line 42: parameters are separated by commas,
line 47: both attributes and methods must finish with a semi colon,
line 48: function readChar() reads just one character, or returns an empty string if the end of file has been reached,
line 53: once the read of file has completed, a message of success is written,
line 55: user-defined function ; may return a value or not. The declaration always starts with keyword function, even if it announces a procedure (no return value). Reading a type is called at several points of the grammar, so the code is factorized in the procedure readType(). It doesn't return any value about success or failure, because an error is thrown in case of syntax mismatch.
line 58: does the keyword is a modifier? If not sType contains a basic type or a class name
line 60: reads the name of the aggregated class
line 64: perhaps that the type is an array, represented by [],
This script seems quite far from the BNF of our simple modeling language, while it implements it in a procedural way. It is able to read a well-formed design file, as our solar system presented at the beginning of the chapter. It doesn't care about populating a parse tree yet, but produces contextual error messages when the design file doesn't conform to the BNF.

Let apply the script on the design file:


parseFree("GettingStarted/SimpleML-token-reading.cws",
        project, "GettingStarted/SolarSystem0.sml");


the file has been read successfully

Now, let improve the script to allow populating a parse tree:

      // file "GettingStarted/SimpleML-token-parsing.cws":
      1 declare function readType(myType : node);
      2
      3 while skipEmptyCpp() {
      4     if !readIfEqualToIdentifier("class") error("'class' expected");
      5     skipEmptyCpp();
      6     local sClassName = readIdentifier();
      7     if !sClassName error("class name expected");
      8     insert project.listOfClasses[sClassName].name = sClassName;
      9     skipEmptyCpp();
      10     if readIfEqualTo(":") {
      11         skipEmptyCpp();
      12         local sParentName = readIdentifier();
      13         if !sParentName error("parent name expected for class '" + sClassName + "'");
      14         insert project.listOfClasses[sClassName].parent = sParentName;
      15         skipEmptyCpp();
      16     }
      17     if !readIfEqualTo("{") error("'{' expected");
      18     skipEmptyCpp();
      19     local myClass;
      20     ref myClass = project.listOfClasses[sClassName];
      21     while !readIfEqualTo("}") {
      22         skipEmptyCpp();
      23         local myType;
      24         readType(myType);
      25         skipEmptyCpp();
      26         local sMemberName = readIdentifier();
      27         if !sMemberName error("attribute or method name expected");
      28         skipEmptyCpp();
      29         if readIfEqualTo("(") {
      30             insert myClass.listOfMethods[sMemberName].name = sMemberName;
      31             if myType.name != "void" {
      32                 setall myClass.listOfMethods[sMemberName].type = myType;
      33             }
      34             skipEmptyCpp();
      35             if !readIfEqualTo(")") {
      36                 local myMethod;
      37                 ref myMethod = myClass.listOfMethods[sMemberName];
      38                 do {
      39                     skipEmptyCpp();
      40                     local iPosition = getInputLocation();
      41                     local sMode = readIdentifier();
      42                     if !sMode error("parameter type or mode expected");
      43                     if (sMode != "in") && (sMode != "out") && (sMode != "inout") {
      44                         setInputLocation(iPosition);
      45                         set sMode = "";
      46                     }
      47                     skipEmptyCpp();
      48                     local myParameterType;
      49                     readType(myParameterType);
      50                     skipEmptyCpp();
      51                     local sParameterName = readIdentifier();
      52                     if !sParameterName error("parameter name expected");
      53                     insert myMethod.listOfParameters[sParameterName].name = sParameterName;
      54                     setall myMethod.listOfParameters[sParameterName].type = myParameterType;
      55                     if sMode {
      56                         insert myMethod.listOfParameters[sParameterName].name = sMode;
      57                     }
      58                     skipEmptyCpp();
      59                 } while readIfEqualTo(",");
      60                 if !readIfEqualTo(")") error("')' expected");
      61             }
      62             skipEmptyCpp();
      63         } else {
      64             insert myClass.listOfAttributes[sMemberName].name = sMemberName;
      65             setall myClass.listOfAttributes[sMemberName].type = myType;
      66         }
      67         if !readIfEqualTo(";") error("';' expected to close an attribute, instead of '" + readChar() + "'");
      68         skipEmptyCpp();
      69     }
      70 }
      71 traceLine("the file has been parsed successfully");
      72
      73 function readType(myType : node) {
      74     local sType = readIdentifier();
      75     if !sType error("type modifier or name expected, instead of '" + readChar() + "'");
      76     if sType == "aggregate" {
      77         insert myType.isAggregation = true;
      78         skipEmptyCpp();
      79         sType = readIdentifier();
      80         if !sType error("aggregated class name expected");
      81     }
      82     insert myType.name = sType;
      83     if (sType != "int") && (sType != "double") && (sType != "boolean") && (sType != "string") {
      84         insert myType.isObject = true;
      85     }
      86     skipEmptyCpp();
      87     if readIfEqualTo("[") {
      88         skipEmptyCpp();
      89         if !readIfEqualTo("]") error("']' expected to close an array declaration");
      90         insert myType.isArray = true;
      91     }
      92 }

line 8: about parsing, classes are modeled into node project.listOfClasses[sClassName]. Its attribute name contains the value of sClassName.
line 14: this class inherits from a parent, so the optional attribute parent of the class is populated with the value of sParentName,
line 19: to work easier with the current class node project.listOfClasses[sClassName], we define a reference to it, called myClass,
line 23: the class is populated with the characteristics of the member once its declaration has finished. Otherwise, it may confuse between an attribute or a method declaration. So, we should have factorized the type declaration and the name of the member into a common clause, for example.
line 30: about parsing, methods are modeled into node myClass.listOfMethods[sMemberName],
line 31: attribute name is compulsory into a type node, so if myType.name returns "void", there is no return type,
line 36: to work easier with the current class node myClass.listOfMethods[sMemberName], we define a reference to it, called myMethod,
line 53: about parsing, parameters are modeled into node myMethod.listOfParameters[sParameterName],
line 64: about parsing, attributes are modeled into node myClass.listOfAttributes[sMemberName],
line 65: the type is allocated on the stack, so it is copied into branch type (no node reference) integrally,
line 71: once the parsing of file has achieved, a message of success is written,
line 73: function readType() requires a node into which description of type will be populated,
line 77: about parsing, myType.isAggregation contains true if type is an array,
line 82: about parsing, myType.name contains the name of basic type,
line 83: check whether the type is a basic one or a class specifier,
line 84: about parsing, myType.isObject contains true because we suppose that this type is a class specifier (by default: it isn't a basic type),
line 90: about parsing, myType.isArray contains true if type is an array,
The first version of the script was just able to read a well-formed design file written in the simple modeling language. The second version validates the file and populates the parse tree:


parseFree("GettingStarted/SimpleML-token-parsing.cws",
        project, "GettingStarted/SolarSystem0.sml");


the file has been parsed successfully

The next script proposes to check the existence of each class specifier types and to keep a reference to the node that describes this class specifier. Some nodes change their nature (myClass.parent becomes a reference to the parent node, for example), some other are added (for object types, the new node myType.class keeps a reference to the class):

      // file "GettingStarted/TreeDecoration.cws":
      1 foreach myClass in project.listOfClasses {
      2     if myClass.parent {
      3         if !findElement(myClass.parent, project.listOfClasses)
      4             error("class '" + myClass.parent + "' doesn't exist while class '"
      5                   + myClass.name + "intends to inherit from it");
      6         ref myClass.parent = project.listOfClasses[myClass.parent];
      7     }
      8     foreach myAttribute in myClass.listOfAttributes {
      9         local myType;
      10         ref myType = myAttribute.type;
      11         if myType.isObject {
      12             if !findElement(myType.name, project.listOfClasses)
      13                 error("class '" + myType.name + "' doesn't exist while attribute '"
      14                       + myClass.name + "::" + myAttribute.name + "' refers to it");
      15             ref myType.class = project.listOfClasses[myType.name];
      16         }
      17     }
      18     foreach myMethod in myClass.listOfMethods {
      19         if existVariable(myMethod.type) && myMethod.type.isObject {
      20             localref myType = myMethod.type;
      21             if !findElement(myType.name, project.listOfClasses)
      22                 error("class '" + myType.name + "' doesn't exist while method '"
      23                       + myClass.name + "::" + myMethod.name + "' refers to it");
      24             ref myType.class = project.listOfClasses[myType.name];
      25         }
      26         foreach myParameter in myMethod.listOfParameters {
      27             localref myType = myParameter.type;
      28             if myType.isObject {
      29                 if !findElement(myType.name, project.listOfClasses)
      30                     error("class '" + myType.name
      31                           + "' doesn't exist while method '"
      32                           + myClass.name + "::" + myMethod.name
      33                           + "' refers to it");
      34                 ref myType.class = project.listOfClasses[myType.name];
      35             }
      36         }
      37     }
      38 }

line 1: we iterate all classes,
line 2: if field parent is filled, we check its existence and then, we change it as a reference to the parent class,
line 8: we iterate all attributes of each class,
line 11: only object attributes are interesting,
line 12: check whether the class exists or not into the array node that contains all classes: does the key myType.name exist as an array entry of node project.listOfClasses?
line 15: to optimize navigating into the parse tree later, we keep a reference to the class into new node myType.class,
line 18: we iterate all methods of each class,
line 26: we iterate all parameters of each method,
Now, we dispose of a parsing script that loads well-formed Simple-Modeling designs, and a script that decorates the parse tree. It is time to write a leader script that will take in charge calling tasks of parsing, tree decoration and source code generation:

CodeWorker command line to execute:
-I Scripts/Tutorial/GettingStarted -define DESIGN_FILE=SolarSystem0.sml -script LeaderScript0.cws

      // file "GettingStarted/LeaderScript0.cws":
      1 if !getProperty("DESIGN_FILE")
      2     error("'-define DESIGN_FILE=file' expected on the command line");
      3 traceLine("'Simple Modeling' design file to parse = \""
      4           + getProperty("DESIGN_FILE") + "\"");
      5 parseAsBNF("SimpleML-parsing.cwp",
      6            project, getProperty("DESIGN_FILE"));
      7 #include "TreeDecoration.cws"

Such a script begins with a sequence of characters exactly like they must be written into the output file, up to it encounters special character '@' or JSP-like tag '<%'. Then it swaps into script mode, and everything is interpreted as script instructions, up to special character '@' or JSP-like tag '%>' are encountered. Content of the script file is again understood as a sequence of characters to write into the output file, up to the next special character. And it continues swapping from a mode to another...

For convenience, the script mode may be just restrained to an expression (often the name of a variable) whose value is written into the output file.

To do source code generation, we'll need some useful functions, such as converting a Simple-Modeling type to its C++ representation. These functions might be included into the leader script, so as to be shared by all pattern scripts.

We'll discover a new type of functions, called template functions that bring a little generic programming in the language: let imagine that we need function getType(myType : node), to decline for every language we could have to generate (C++ and JAVA in this chapter). You plan to generate an object library from the design you have written in the Simple Modeling Language. This object library will be delivered both in C++ and JAVA, and a technical documentation will come with each of these implementations. This technical documentation will give the signature of methods and the type of attributes in the language the developer will choose. So the C++ documentation will be slightly different from the JAVA one, just at the level of type's spelling. Normally, you'll write the following lines to recover the type depending on the language for which you are producing the documentation:


if doc_language == "C++" {
    sType = getCppType(myParameterType);
} else if doc_language == "JAVA" {
    sType = getJAVAType(myParameterType);
} else {
    error("unrecognized language '" + doc_language + "'");
}



sType = getType<doc_language>(myParameterType);
...
function getType<"JAVA">(myType : node) {
    ... // implementation for returning a Java type
}

function getType<"C++">(myType : node) {
    ... // implementation for returning a C++ type
}


Trying to call an instantiated function that doesn't exist, raises an error at runtime. However, one might imagine an implementation by default. For instance:


function getType<T>(myType : node) {
    ... // implementation for any unrecognized language
}


For those that know generic programming with C++ templates, here is a classical example of using template functions:


function f<1>() { return 1; }
function f<N>() { return $N*f<$N - 1$>()$; }
local f10 = f<10>();
if $f10 != 3628800$ error("10! should be worth 3628800");
traceLine("10! = " + f10);


10! = 3628800

We'll find below all useful functions we'll need for source code generation, including the template function getType<T>(myType : node) we spoke about:

      // file "GettingStarted/SharedFunctions.cws":
      1 function normalizeIdentifier(sName) {
      2     if sName {
      3         if startString(sName, "_")
      4             return "_" + normalizeIdentifier(subString(sName, 1));
      5         set sName = toUpperString(charAt(sName, 0))
      6                     + subString(sName, 1);
      7         local iIndex = findFirstChar(sName, "_.");
      8         if !isNegative(iIndex) {
      9             local sNext = subString(sName, add(iIndex, 1));
      10             return leftString(sName, iIndex)
      11                     + normalizeIdentifier(sNext);
      12         }
      13     }
      14     return sName;
      15 }
      16
      17 function getType<"C++">(myType : node) {
      18     local sType;
      19     if myType.isObject set sType = myType.name + "*";
      20     else if myType.name == "boolean" set sType = "bool";
      21     else if myType.name == "string" set sType = "std::string";
      22     else set sType = myType.name;
      23     if myType.isArray set sType = "std::vector<" + sType + ">";
      24     return sType;
      25 }
      26
      27 function getParameterType<"C++">(myType : node, sMode) {
      28     local sType = getType<"C++">(myType);
      29     if endString(sMode, "out") set sType += "&";
      30     else if (sMode == "in") set sType = "const " + sType + "&";
      31     return sType;
      32 }
      33
      34 function getType<"JAVA">(myType : node) {
      35     local sType;
      36     if myType.name == "string" set sType = "String";
      37     else set sType = myType.name;
      38     if myType.isArray set sType = "java.util.ArrayList/*<" + sType + ">*/";
      39     return sType;
      40 }
      41
      42 function getParameterType<"JAVA">(myType : node, sMode) {
      43     return getType<"JAVA">(myType);
      44 }
      45
      46 function getVariableName(sName, myType : node) {
      47     local sPrefix;
      48     if myType.isArray set sPrefix = "t";
      49     if myType.isObject set sPrefix += "p";
      50     else {
      51         switch(myType.name) {
      52             case "int": set sPrefix += "i";break;
      53             case "double": set sPrefix += "d";break;
      54             case "boolean": set sPrefix += "b";break;
      55             case "string": set sPrefix += "s";break;
      56         }
      57     }
      58     return sPrefix + normalizeIdentifier(sName);
      59 }
      60
      61 function getMethodID(myMethod : node) {
      62     local sMethodID = myMethod.name;
      63     foreach i in myMethod.listOfParameters {
      64         set sMethodID += "." + i.type.name;
      65         if i.type.isArray set sMethodID += "[]";
      66     }
      67     return sMethodID;
      68 }

line 1: this function normalizes identifiers, so as to capitalize the first letter and to suppress '_' or dots after capitalizing the letter that follows: average_speed becomes AverageSpeed, for example. This function is applied on attribute names for instance.
line 3: if the identifier starts with an underscore, it is preserved,
line 7: points to the first character encountered among an underscore and a dot,
line 17: this function returns the C++ type of a Simple-Modeling type node:

• an object returns a pointer to it,
• type boolean is written bool in C++,
• type string is written std::string in the C++ standard library,
• an array is written as an instantiated class of std::vector,

line 27: this function returns the C++ type of a Simple-Modeling type node as expected when passed to a method as a parameter type (sMode is worth "in", "out", "inout" or empty string),
line 34: this function returns the JAVA type of a Simple-Modeling type node:

• an object returns its class name,
• type boolean is written identically in JAVA,
• type string is written String in JAVA,
• an array is written as a java.util.ArrayList interface in JAVA,

line 42: this function returns the JAVA type of a Simple-Modeling type node as expected when passed to a method as a parameter type (sMode is worth "in", "out", "inout" or empty string, but we don't care about "inout" or "out" for the moment),
line 46: this function returns a variable name whose nomenclature depends on its type,
line 51: the switch statement allows selection among multiple sections of code, depending on the value of expression myType.name, enclosed in parentheses. If no controlling expression (announced by label case) matches with the value, and no default label is present, CodeWorker throws an error.
line 61: this function returns a unique method ID, which is composed from the name of the method and the type of parameters, to avoid confusing protected areas from a method to another,
The next two examples both implement same functionalities, but in different languages (C++ and JAVA). They describe the skeleton of our objects.

• the first one is the file name of the script,
• the second one is the current context of execution that will be accessed via the this keyword into the script,
• the last one is the name of the file to generate,

The next pattern script describes the pattern of a C++ header file:

      // file "GettingStarted/CppObjectHeader.cwt":
      1 #ifndef _@this.name@_h_
      2 #define _@this.name@_h_
      3
      4 @
      5 newFloatingLocation("include files");
      6 @
      7 // this line separates the two insertion points, so as to distinguish them!
      8 @
      9 newFloatingLocation("class declarations");
      10
      11 function populateHeaderDeclarations(myType : node) {
      12     if myType.isObject insertTextOnce(getFloatingLocation("class declarations"), "class " + myType.name + ";" + endl());
      13     if myType.isArray insertTextOnce(getFloatingLocation("include files"), "#include <vector>" + endl());
      14     if myType.name insertTextOnce(getFloatingLocation("include files"), "#include <string>" + endl());
      15 }
      16
      17 @
      18 class @this.name@ @
      19 if existVariable(this.parent) {
      20     insertTextOnce(getFloatingLocation("include files"), "#include \"" + this.parent.name +".h\"" + endl());
      21     @: public @this.parent.name@ @
      22 }
      23 @{
      24     private:
      25 @
      26 foreach i in this.listOfAttributes {
      27     populateHeaderDeclarations(i.type);
      28     @ @getType<"C++">(i.type)@ _@getVariableName(i.name, i.type)@;
      29 @
      30 }
      31 @
      32     public:
      33         @this.name@();
      34         ~@this.name@();
      35
      36         // accessors:
      37 @
      38 foreach i in this.listOfAttributes {
      39     local sVariableName = getVariableName(i.name, i.type);
      40     %> inline <%getType<"C++">(i.type)%> get<%normalizeIdentifier (i.name)%>() const { return _<%sVariableName%>; }
      41         inline void set<%normalizeIdentifier(i.name)@(<%getType <"C++">(i.type)%> <%sVariableName@) { _<%sVariableName%> = <%sVariableName%>; }
      42 @
      43 }
      44 @
      45         // methods:
      46 @
      47 foreach i in this.listOfMethods {
      48     @ virtual @
      49     if existVariable(i.type) {
      50         populateHeaderDeclarations(i.type);
      51         @@getType<"C++">(i.type)@@
      52     } else {
      53         @void@
      54     }
      55     @ @i.name@(@
      56     foreach j in i.listOfParameters {
      57         if !first(j) {
      58             @, @
      59         }
      60         populateHeaderDeclarations(j.type);
      61         @@getParameterType<"C++">(j.type, j.mode)@ @getVariableName(j.name, j.type)@@
      62     }
      63     @);
      64 @
      65 }
      66 @
      67     private:
      68         @this.name@(const @this.name@&);
      69         @this.name@& operator =(const @this.name@&);
      70 };
      71
      72 #endif

line 1: the value of attribute this.name is written to the output file, where this points to a node that describes the current class. Note that this is facultative, and is assigned by the caller of procedure generate that runs this script.
line 5: put one anchor for including all files that we'll encounter as compulsory, while iterating attributes or methods. Example: if an attribute is an array, we'll need to include the STL header vector at this position of the file: #include <vector>. This insertion point is called "include files".
line 6: to avoid that the two floating locations "include files" and "class declarations" (described just below) point to the same file position, an empty line is added,
line 9: put one anchor for announcing all classes that we'll encounter as referenced, while iterating attributes or methods. Example: if an attribute is an object Planet, we'll need to write class Planet; at this position of the file. This insertion point is called "class declarations".
line 11: this function is called on every type encountered while iterating attributes and methods. Its role is to populate the "include files" and "class declarations" areas.
line 12: the type of an object must be declared at the beginning of the header, otherwise the compiler will not recognize it : the class is declared once only in the insertion point called "class declarations". Use of function insertTextOnce assures that if this class has already been inserted before, it will not be twice.
line 13: this type is an array, so the declaration of std::vector must be included to the insertion point called "include files",
line 14: this type is a string, so the declaration of std::string must be included to the insertion point called "include files",
line 19: if the class inherits from a parent class, this relationship must be written,
line 20: the parent class must be declared,
line 26: declaration of all attributes,
line 27: does the type of the attribute need some backward declarations?
line 38: accessors to each attribute,
line 40: there are two symbols to swap between writing a sequence of characters and interpreting script ; we have used the symbol '@', and now we illustrate the use of tags '<% and '%>,
line 41: you can melt the two swapping symbol, but it is more difficult to read, so not very interesting!
line 47: declaration of all methods,
line 48: each method might be overloaded by subclasses,
line 49: the return type of the method is translated to C++,
line 50: does the return type of the method need some backward declarations?
line 51: expression getType<"C++">(i.type) to evaluate is embedded between double '@'. The first one allow swapping to the sequence of characters mode, but there is no characters to write. The second one allows swapping to the script mode, which is reduced just to evaluate the expression. The two final '@' take the same role as seen before.
line 56: parameters of the method are iterated to be written in C++
line 57: if iterator j doesn't point to the first parameter, a comma makes a separation with the precedent,
line 60: does the type of the parameter need some backward declarations?
Let's continue with the pattern that describes the skeleton of a C++ body file:

      // file "GettingStarted/CppObjectBody.cwt":
      1 #ifdef WIN32
      2 #pragma warning(disable : 4786)
      3 #endif
      4
      5 @
      6 setProtectedArea("include files");
      7 @
      8 #include "@this.name@.h"
      9
      10 @this.name@::@this.name@()@
      11 local bAtLeastOne = false;
      12 foreach i in this.listOfAttributes {
      13     if !i.type.isArray && (i.type.name != "string") {
      14         if bAtLeastOne {
      15             @, @
      16         } else {
      17             @ : @
      18             set bAtLeastOne = true;
      19         }
      20         @_@getVariableName(i.name, i.type)@(@
      21         if i.type.isObject {
      22             @0L@
      23         } else {
      24             switch(i.type.name) {
      25                 case "int":
      26                     @0@
      27                     break;
      28                 case "double":
      29                     @0.0@
      30                     break;
      31                 case "boolean":
      32                     @false@
      33                     break;
      34             }
      35         }
      36         @)@
      37     }
      38 }
      39 @ {
      40 }
      41
      42 @this.name@::~@this.name@() {
      43 @
      44 foreach i in this.listOfAttributes {
      45     if i.type.isAggregation && i.type.isObject {
      46         local sAttributeName = "_" + getVariableName(i.name, i.type);
      47         local sIndex = "iterate" + normalizeIdentifier(i.name);
      48         if i.type.isArray {
      49             @ for (std::vector<@i.name@*>::const_iterator @sIndex@ = @sAttributeName@.begin(); @sIndex@ != @sAttributeName@.end(); ++@sIndex@) {
      50         delete *@sIndex@;
      51     }
      52 @
      53         } else {
      54             @ delete @sAttributeName@;
      55 @
      56         }
      57     }
      58 }
      59 @}
      60
      61 @
      62 foreach i in this.listOfMethods {
      63     if existVariable(i.type) {
      64         @@getType<"C++">(i.type)@@
      65     } else {
      66         @void@
      67     }
      68     @ @this.name@::@i.name@(@
      69     foreach j in i.listOfParameters {
      70         if !first(j) {
      71             @, @
      72         }
      73         @@getParameterType<"C++">(j.type, j.mode)@ @getVariableName(j.name, j.type)@@
      74     }
      75     @) {
      76 @
      77         setProtectedArea(getMethodID(i));
      78 @}
      79 @
      80 }

line 1: Visual C++-specific pragma must be added to prevent from intempestive warnings about template class instantiation of std::vector<T> in DEBUG mode!
line 6: the developer will add here all include files he will need for implementation of methods,
line 8: the header of this body is compulsory,
line 11: this part concerns the initialization of attributes. Some attributes, such as strings and vectors of the STL don't require to be initialized explicitly. It justifies the declaration of variable bAtLeastOne that is worth false as long as no attribute has been initialized yet. We'll see why below.
line 13: arrays and strings are skipped,
line 15: if it isn't the first attribute to be initialized, a comma make a separation with the precedent,
line 17: if it is the first attribute to be initialized, a colon is expected to announce the beginning of initializations
line 18: now, there is at least one attribute to be initialized,
line 21: attribute is populated with the default value corresponding to its type,
line 44: aggregated objects must be deleted before leaving this instance,
line 49: all elements of an aggregated array must be deleted
line 54: the aggregated object is deleted
line 62: implementation of all methods,
line 63: the return type of the method is translated to C++,
line 69: parameters of the method are iterated to be written in C++
line 70: if iterator j doesn't point to the first parameter, a comma makes a separation with the precedent,
line 77: a protected area is inserted, whose key is the method ID,
The leader script has to be improved to reclaim generation of C++ files:

CodeWorker command line to execute:
-I Scripts/Tutorial -path . -define DESIGN_FILE=GettingStarted/SolarSystem0.sml -script GettingStarted/LeaderScript1.cws

      // file "GettingStarted/LeaderScript1.cws":
      1 if !getProperty("DESIGN_FILE")
      2     error("'-define DESIGN_FILE=file' expected on the command line");
      3 traceLine("'Simple Modeling' design file to parse = \""
      4           + getProperty("DESIGN_FILE") + "\"");
      5 parseAsBNF("GettingStarted/SimpleML-parsing.cwp",
      6            project, getProperty("DESIGN_FILE"));
      7 #include "TreeDecoration.cws"
      8
      9 #include "SharedFunctions.cws"
      10 foreach myClass in project.listOfClasses {
      11     traceLine("generating class '" + myClass.name + "' ...");
      12     generate("GettingStarted/CppObjectHeader.cwt", myClass,
      13              getWorkingPath() + "Scripts/Tutorial/GettingStarted/Cpp/"
      14              + myClass.name + ".h");
      15     generate("GettingStarted/CppObjectBody.cwt", myClass,
      16              getWorkingPath() + "Scripts/Tutorial/GettingStarted/Cpp/"
      17              + myClass.name + ".cpp");
      18 }

line 9: all useful functions for source code generation are loaded here,
line 10: all classes are iterated and their C++ header and body are generated
line 12: instruction generate is applied on a pattern script and its second argument expects a node that will be seen as variable 'this' into the pattern script,
line 13: getWorkingPath() is worth the output path passed to the command line via the option '-path',

Output:

'Simple Modeling' design file to parse = "GettingStarted/SolarSystem0.sml"
file parsed successfully
generating class 'Planet' ...
generating class 'Earth' ...
generating class 'SolarSystem' ...

Let have a look on some generated files:

      // file "GettingStarted/Cpp/SolarSystem.h":
      #ifndef _SolarSystem_h_
      #define _SolarSystem_h_
     
      #include <vector>
      #include <string>
     
      // this line separates the two insertion points, so as to distinguish them!
      class Planet;
     
      class SolarSystem {
          private:
              std::vector<Planet*> _tpPlanets;
     
          public:
              SolarSystem();
              ~SolarSystem();
     
              // accessors:
              inline std::vector<Planet*> getPlanets() const { return _tpPlanets; }
              inline void setPlanets(std::vector<Planet*> tpPlanets) { _tpPlanets = tpPlanets; }
     
              // methods:
     
          private:
              SolarSystem(const SolarSystem&);
              SolarSystem& operator =(const SolarSystem&);
      };
     
      #endif

      // file "GettingStarted/Cpp/Planet.cpp":
      1 #ifdef WIN32
      2 #pragma warning(disable : 4786)
      3 #endif
      4
      5 //##protect##"include files"
      6 //##protect##"include files"
      7
      8 #include "Planet.h"
      9
      10 Planet::Planet() : _dDiameter(0.0) {
      11 }
      12
      13 Planet::~Planet() {
      14 }
      15
      16 double Planet::getDistanceToSun(int iDay, int iMonth, int iYear) {
      17 //##protect##"getDistanceToSun.int.int.int"
      18 //##protect##"getDistanceToSun.int.int.int"
      19 }

Our design is totally independent from the implementation : a string isn't explicitly a const std::string& or a std::string in C++, but the pattern script decides according to the context whether it is more judicious to choose the first C++ mapping or the second one.

This independence allows us implementing the same functionalities as in C++, but in JAVA now:

      // file "GettingStarted/JAVAObject.cwt":
      1 package solarsystem;
      2
      3 public class @this.name@ @
      4 if existVariable(this.parent) {
      5     @extends @this.parent.name@ @
      6 }
      7 @{
      8 @
      9 foreach i in this.listOfAttributes {
      10     @ private @getType<"JAVA">(i.type)@ _@getVariableName(i.name, i.type)@;
      11 @
      12 }
      13 @
      14     public @this.name@() {}
      15
      16     // accessors:
      17 @
      18 foreach i in this.listOfAttributes {
      19     local sVariableName = getVariableName(i.name, i.type);
      20     @ public @getType<"JAVA">(i.type)@ get@normalizeIdentifier(i.name)@() { return _@sVariableName@; }
      21     public void set@normalizeIdentifier(i.name)@(@getType<"JAVA">(i.type)@ @sVariableName@) { _@sVariableName@ = @sVariableName@; }
      22 @
      23 }
      24 @
      25         // methods:
      26 @
      27 foreach i in this.listOfMethods {
      28     @ public @
      29     if existVariable(i.type) {
      30         @@getType<"JAVA">(i.type)@@
      31     } else {
      32         @void@
      33     }
      34     @ @i.name@(@
      35     foreach j in i.listOfParameters {
      36         if !first(j) {
      37             @, @
      38         }
      39         @@getParameterType<"JAVA">(j.type, j.mode)@ @getVariableName(j.name, j.type)@@
      40     }
      41     @) {
      42 @
      43     setProtectedArea(getMethodID(i));
      44 @ }
      45
      46 @
      47 }
      48 @}

line 4: if the class inherits from a parent class, this relationship must be written,
line 9: declaration of all attributes,
line 18: accessors to each attribute,
line 27: declaration of all methods,
The leader script has to be improved to reclaim generation of JAVA files:

CodeWorker command line to execute:
-I Scripts/Tutorial -path . -define DESIGN_FILE=GettingStarted/SolarSystem0.sml -script GettingStarted/LeaderScript2.cws

      // file "GettingStarted/LeaderScript2.cws":
      1 if !getProperty("DESIGN_FILE")
      2     error("'-define DESIGN_FILE=file' expected on the command line");
      3 traceLine("'Simple Modeling' design file to parse = \""
      4           + getProperty("DESIGN_FILE") + "\"");
      5 parseAsBNF("GettingStarted/SimpleML-parsing.cwp",
      6            project, getProperty("DESIGN_FILE"));
      7 #include "TreeDecoration.cws"
      8
      9 #include "SharedFunctions.cws"
      10 foreach myClass in project.listOfClasses {
      11     traceLine("generating class '" + myClass.name + "' ...");
      12     generate("GettingStarted/CppObjectHeader.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/Cpp/" + myClass.name + ".h");
      13     generate("GettingStarted/CppObjectBody.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/Cpp/" + myClass.name + ".cpp");
      14     generate("GettingStarted/JAVAObject.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/JAVA/solarsystem/" + myClass.name + ".java");
      15 }

line 14: generates the JAVA implementation of the current design class,

Output:

'Simple Modeling' design file to parse = "GettingStarted/SolarSystem0.sml"
file parsed successfully
generating class 'Planet' ...
generating class 'Earth' ...
generating class 'SolarSystem' ...

Let have a look on some generated files:

      // file "GettingStarted/JAVA/solarsystem/SolarSystem.java":
      package solarsystem;
     
      public class SolarSystem {
          private java.util.ArrayList/*<Planet>*/ _tpPlanets;
     
          public SolarSystem() {}
     
          // accessors:
          public java.util.ArrayList/*<Planet>*/ getPlanets() { return _tpPlanets; }
          public void setPlanets(java.util.ArrayList/*<Planet>*/ tpPlanets) { _tpPlanets = tpPlanets; }
     
              // methods:
      }

      // file "GettingStarted/JAVA/solarsystem/Planet.java":
      package solarsystem;
     
      public class Planet {
          private double _dDiameter;
     
          public Planet() {}
     
          // accessors:
          public double getDiameter() { return _dDiameter; }
          public void setDiameter(double dDiameter) { _dDiameter = dDiameter; }
     
              // methods:
          public double getDistanceToSun(int iDay, int iMonth, int iYear) {
      //##protect##"getDistanceToSun.int.int.int"
      //##protect##"getDistanceToSun.int.int.int"
          }
     
      }

For example, a valid markup inlayed in a C++ file could be:
//##markup##"factory"
and a valid markup inlayed in an HTML file could be:
<!- -##markup##"classes"- ->

Up to now, we discovered parsing on one side and source code generation on the other side. The translation mode merges the two: it offers to parse a file conforming to a BNF and to translate it into another format, all in the same translation script.

A translation script looks like a BNF-driven parsing script, but where:

• special swapping character '@' or JSP-like tag '<%',
• all functions and procedure intended to source code generation,

Outputs are written into another file, so the input file is preserved. The procedure that takes the translation in charge is called translate().

Little practical example: all our documentation has been written in HTML, but we would like to translate it to LaTeX, into our own format. Why not?

First step, we must be able to read an HTML file according to a BNF representation. The corresponding BNF-driven script we have to write is restricted to be able to write our file "SolarSystem1.html":

      // file "GettingStarted/HTML-parsing.cwp":
      1 #noCase
      2
      3 HTML ::= #ignore(HTML) #continue '<' "HTML" '>' HTMLHeader HTMLBody '<' '/' "HTML" '>' #empty;
      4 HTMLHeader ::= '<' #continue "HEAD" '>' [~['<' '/' "HEAD" '>']]* '<' '/' "HEAD" '>';
      5 HTMLBody ::= '<' #continue "BODY" '>' HTMLText '<' '/' "BODY" '>';
      6 HTMLText ::=
      7         [
      8             ~'<'
      9                 |
      10             !['<' '/'] #continue '<'
      11                 #readIdentifier:sTag HTMLNextOfTag<sTag>
      12         ]*;
      13 HTMLNextOfTag<"H1"> ::= #continue '>' HTMLText '<' '/' "H1" '>';
      14 HTMLNextOfTag<"H2"> ::= #continue '>' HTMLText '<' '/' "H2" '>';
      15 HTMLNextOfTag<"A"> ::= [HTMLAttribute]* #continue '>' HTMLText '<' '/' 'A' '>';
      16 HTMLNextOfTag<"TABLE"> ::= [HTMLAttribute]* #continue '>' [HTMLTag("TR")]* '<' '/' "TABLE" '>';
      17 HTMLTag(sTag : value) ::= '<' #readText(sTag) #continue HTMLNextOfTag<sTag>;
      18 HTMLNextOfTag<"TR"> ::= [HTMLAttribute]* #continue '>' [HTMLTag("TD")]* '<' '/' "TR" '>';
      19 HTMLNextOfTag<"TD"> ::= [HTMLAttribute]* #continue '>' HTMLText '<' '/' "TD" '>';
      20 HTMLNextOfTag<"UL"> ::= [HTMLAttribute]* #continue '>' [HTMLTag("LI")]* '<' '/' "UL" '>';
      21 HTMLNextOfTag<"LI"> ::= [HTMLAttribute]* #continue '>' HTMLText '<' '/' "LI" '>';
      22 HTMLNextOfTag<"B"> ::= #continue '>' HTMLText '<' '/' "B" '>';
      23 HTMLNextOfTag<"I"> ::= #continue '>' HTMLText '<' '/' "I" '>';
      24 HTMLNextOfTag<"FONT"> ::= [HTMLAttribute]* #continue '>' HTMLText '<' '/' "FONT" '>';
      25 HTMLNextOfTag<"BR"> ::= ['/']? #continue '>';
      26 HTMLAttribute ::= #readIdentifier ['=' #continue [STRING_LITERAL | WORD_LITERAL]]?;
      27
      28
      29 STRING_LITERAL ::= #!ignore '\"' [~'\"']* '\"';
      30 WORD_LITERAL ::= #!ignore [~['>' | '/' | ' ' | '\t']]+;

line 1: we don't care about the case: <BODY> and <Body> must be recognized as identical for instance,
line 6: the clause HTMLText reads the value between tags,
line 11: the best way to assure an easy extension of the grammar: to declare a template clause for describing the reading of a tag,
line 17: a clause to read a determined tag: the token #readText matches the input stream to the evaluated expression passed in parameter and the rest is read by the template clause that describes the reading of a tag,
Second step, we have to improve the BNF-driven script to add some features for generating the LaTeX code properly. Don't be afraid about the length of the source code, but go forward to the notes directly:

      // file "GettingStarted/HTML2LaTeX.cwp":
      1 #noCase
      2
      3 HTML2LaTeX ::= #ignore(HTML) #continue '<' "HTML" '>' HTMLHeader HTMLBody '<' '/' "HTML" '>' #empty;
      4 HTMLHeader ::= '<' #continue "HEAD" '>' [~['<' '/' "HEAD" '>']]* '<' '/' "HEAD" '>';
      5 HTMLBody ::= '<' #continue "BODY" '>' HTMLText '<' '/' "BODY" '>';
      6 HTMLText ::= #!ignore
      7         [
      8             '&' #continue #readIdentifier:sEscape HTMLEscape<sEscape> ';'
      9                 |
      10             ~'<':cChar => writeText(cChar);
      11                 |
      12             !['<' blanks '/']
      13             [
      14                 "<!--" #continue [~"-->"]* "-->"
      15                     |
      16                 '<' #continue #ignore(HTML) #readIdentifier:sTag HTMLNextOfTag<sTag>
      17             ]
      18         ]*;
      19 HTMLEscape<"lt"> ::= => {@<@};
      20 HTMLEscape<"gt"> ::= => {@>@};
      21 HTMLTag(sTag : value) ::= '<' #readText(sTag) #continue HTMLNextOfTag<sTag>;
      22 HTMLNextOfTag<"H1"> ::=
      23         #continue '>' => {@\subsection{@}
      24         HTMLText
      25         '<' '/' "H1" '>' => {@}@};
      26 HTMLNextOfTag<"H2"> ::=
      27         #continue '>' => {@\subsubsection{@}
      28         HTMLText
      29         '<' '/' "H2" '>' => {@}@};
      30 HTMLNextOfTag<"A"> ::= [HTMLAttribute]* #continue '>' HTMLText '<' '/' 'A' '>';
      31 HTMLNextOfTag<"TABLE"> ::=
      32         [HTMLAttribute]* #continue '>' => {
      33             @\begin{table@
      34             newFloatingLocation("table PDF suffix");
      35             @}{@
      36             newFloatingLocation("table columns");
      37             @}{.5}@
      38         }
      39         => local sPDFTableSuffix;
      40         HTMLTableTitle(sPDFTableSuffix)
      41         [HTMLTableLine(sPDFTableSuffix)]*
      42         '<' '/' "TABLE" '>' => {@\end{table@sPDFTableSuffix@}
      43 @};
      44 HTMLTableTitle(sPDFTableSuffix : node) ::=
      45     '<' "TR" [HTMLAttribute]*
      46     #continue '>'
      47     [HTMLTableCol(sPDFTableSuffix)]*
      48     '<' '/' "TR" '>' => {
      49         insertText(getFloatingLocation("table PDF suffix"), sPDFTableSuffix);
      50         writeText(endl());
      51     };
      52 HTMLTableCol(sPDFTableSuffix : node) ::=
      53     '<' "TD" [HTMLAttribute]* #continue '>' => {
      54         @{@
      55         if !sPDFTableSuffix insertText(getFloatingLocation("table columns"), "l");
      56         else insertText(getFloatingLocation("table columns"), "|l");
      57         set sPDFTableSuffix += "i";
      58     }
      59     '<' 'B' '>' [#!ignore [~'<':cChar => writeText(cChar);]*] '<' '/' 'B' '>'
      60     '<' '/' "TD" '>' => {@}@};
      61 HTMLTableLine(sPDFTableSuffix : value) ::=
      62         '<' "TR" [HTMLAttribute]* #continue '>' => {@\line@sPDFTableSuffix@@}
      63         [HTMLTag("TD")]* '<' '/' "TR" '>' => {writeText(endl());};
      64 HTMLNextOfTag<"TD"> ::=
      65         [HTMLAttribute]* #continue '>' => {@{@}
      66         HTMLCellText '<' '/' "TD" '>' => {@}@};
      67 HTMLCellText ::= #!ignore
      68         [
      69             '&' #continue #readIdentifier:sEscape HTMLEscape<sEscape> ';'
      70                 |
      71             ['\r']? ['\n'] => {@ @}
      72                 |
      73             ~'<':cChar => writeText(cChar);
      74                 |
      75             !['<' blanks '/']
      76             [
      77                 "<!--" #continue [~"-->"]* "-->"
      78                     |
      79                 '<' #continue #ignore(HTML) #readIdentifier:sTag HTMLNextOfTag<sTag>
      80             ]
      81         ]*;
      82 HTMLNextOfTag<"UL"> ::=
      83         [HTMLAttribute]* #continue '>' => {@\begin{itemize}
      84 @}
      85         [HTMLTag("LI")]*
      86         '<' '/' "UL" '>' => {@\end{itemize}
      87 @};
      88 HTMLNextOfTag<"LI"> ::=
      89         [HTMLAttribute]* #continue '>' => {@\item @}
      90         HTMLText
      91         '<' '/' "LI" '>' => {writeText(endl());};
      92 HTMLNextOfTag<"B"> ::=
      93         #continue '>' => {@\textbf{@}
      94         HTMLText
      95         '<' '/' "B" '>' => {@}@};
      96 HTMLNextOfTag<"I"> ::=
      97         #continue '>' => {@\textbf{@}
      98         HTMLText
      99         '<' '/' "I" '>' => {@}@};
      100 HTMLNextOfTag<"FONT"> ::= [HTMLAttribute]* #continue '>' HTMLText '<' '/' "FONT" '>';
      101 HTMLNextOfTag<"BR"> ::= ['/']? #continue '>' => { writeText(endl());};
      102 HTMLAttribute ::= #readIdentifier ['=' #continue [STRING_LITERAL | WORD_LITERAL]]?;
      103
      104
      105 blanks ::= [' '| '\t' | '\r' | '\n']*;
      106 STRING_LITERAL ::= #!ignore '\"' [~'\"']* '\"';
      107 WORD_LITERAL ::= #!ignore [~['>' | '/' | ' ' | '\t']]+;

line 6: blank characters are interesting, so we refuse to ignore HTML blanks and comments,
line 8: handling of HTML escape sequences, announced by character '&',
line 10: if not the beginning of a tag, the current character of the input stream is put to the output stream,
line 12: token operator '!' doesn't move the position of the input stream, and it continues in sequence only if the token expression that follows doesn't match; here, we check whether we have reached an end of tag or not,
line 14: we do not ignore comments anymore, so we have to do it my ourselves,
line 16: an embedded tag has been encountered,
line 19: template clauses HTMLEscape<T> are always valid and just convert special characters to their LaTeX representation,
line 22: in the real life, HTML tag <H1> could represent a chapter, but the LaTeX output file is intended to be included into the reference manual of CodeWorker as an illustration ; it will be a part of a section, so chapters are translated as sub sections!
line 26: in the real life, HTML tag <H2> could represent a section, but for the same reason as above, it will be translated as a sub-sub section,
line 34: with HTML, the number of columns the table expects is deduced later. However, a latex table (well-formed for a PDF conversion) must know explicetly of how many columns it is composed. So, a floating position is attached to the current position of the output file. While discovering columns, text will be inserted here and further.
line 36: the format of each column is specified at this place,
line 39: we consider that the first line of the table gives the name of the columns, and we'll take the PDF table suffix ('ii' for 2 columns, 'iii' for 3 columns, ...) to write lines of the table correctly,
line 41: we translate as many lines of the table as we can read, knowing the PDF suffix,
line 52: the clause is intended to read the name of a column of a table, and to translate it to LaTeX, knowing that some text must be inserted into the declarative part of the LaTeX table,
line 67: the text into a cell of a table shouldn't contain paragraph jumps (empty line in LaTeX),
line 71: the simplest way to avoid empty lines is to ignore end of lines, and to replace it to a space,
Last step, we have to change the leader script, so as to take into account the translation of the HTML documentation to the LaTeX one:

CodeWorker command line to execute:
-I Scripts/Tutorial -path . -define DESIGN_FILE=GettingStarted/SolarSystem0.sml -script GettingStarted/LeaderScript4.cws

      // file "GettingStarted/LeaderScript4.cws":
      1 if !getProperty("DESIGN_FILE")
      2     error("'-define DESIGN_FILE=file' expected on the command line");
      3 traceLine("'Simple Modeling' design file to parse = \""
      4           + getProperty("DESIGN_FILE") + "\"");
      5 parseAsBNF("GettingStarted/SimpleML-parsing.cwp",
      6            project, getProperty("DESIGN_FILE"));
      7 #include "TreeDecoration.cws"
      8
      9 #include "SharedFunctions.cws"
      10 foreach myClass in project.listOfClasses {
      11     traceLine("generating class '" + myClass.name + "' ...");
      12     generate("GettingStarted/CppObjectHeader.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/Cpp/" + myClass.name + ".h");
      13     generate("GettingStarted/CppObjectBody.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/Cpp/" + myClass.name + ".cpp");
      14     generate("GettingStarted/JAVAObject.cwt", myClass, getWorkingPath() + "Scripts/Tutorial/GettingStarted/JAVA/solarsystem/" + myClass.name + ".java");
      15 }
      16
      17 local myDocumentationContext;
      18 insert myDocumentationContext.language = "C++";
      19 traceLine("generating the HTML documentation...");
      20 setCommentBegin("<!--");
      21 setCommentEnd("-->");
      22 expand("GettingStarted/HTMLDocumentation.cwt",
      23         myDocumentationContext, getWorkingPath()
      24         + "Scripts/Tutorial/GettingStarted/SolarSystem1.html");
      25 translate("GettingStarted/HTML2LaTeX.cwp", project, "GettingStarted/SolarSystem1.html", getWorkingPath() + "Scripts/Tutorial/GettingStarted/SolarSystem.tex");

line 22: the procedure expand() will allow populating "SolarSystem1.html" with the characteristics of the project,
line 25: a context of execution (project here) is given as a this variable, although no parsing will be processed: reading and writing only, no data to keep,

Output:

'Simple Modeling' design file to parse = "GettingStarted/SolarSystem0.sml"
file parsed successfully
generating class 'Planet' ...
generating class 'Earth' ...
generating class 'SolarSystem' ...
generating the HTML documentation...

This class represents the characteristics of a planet.

• function double getDistanceToSun(int iDay, int iMonth, int iYear)

  This function returns the distance to the sun at a given trivial earthly date. This function reclaims more attributes for the planet, but we'll see it later (I'm afraid not!).

This class represents our planet, for instantiating our particular solar system for instance, and working on geopolitical data perhaps!

This class represents the solar system, with its constituents, the sun excluded for the moment.

Once the scripts are considered as stable, it might be interesting to convert the interpreter and all necessary scripts to an executable, for many reasons:

• the executable is largely faster than the interpreter, especially on big projects,
• a script file could be forgotten while delivering the project to somebody else,
• it is more convenient to handle an executable only rather than a set of script files and a long command line on the interpreter,

CodeWorker must be seen as a script interpreter that is intended to parse and to generate any kind of text or source code. This interpreter admits some options on the command line. Some of them look like those of a compiler.

CodeWorker doesn't provide any Graphical User Interface, but a console mode allows interactivity with the user.

• the script describes all the processing tasks for parsing text, decorating the graph and generating code ; the option of the command line is -script to execute the script,
• the script describes an extended BNF grammar ; the option of the command line is -parseBNF for executing the script and parsing the source file,
• the script describes how to generate code ; the option of the command line is -generate to execute the script and to generate the output file,
• the script describes how to expand a file ; the option of the command line is -expand to execute the script and to expand the output file into its markups,
• a file contains embedded scripts driving their own expansion ; the option of the command line is -autoexpand to execute embedded scripts located below each markups, expanding the output file on markups,
• the script describes a source-to-source translation ; the option of the command line is -translate to execute the script and to translate the source file to the output file,

To find easier a file to open for reading among some directories, the option -I specifies a path to explore. It gives more flexibility in sharing input files (both scripts and user files, excepting generated or expanded files) between directories, and it avoids relative or absolute paths into scripts.

It is possible to define some properties on the command line, thanks to option -define (or -D). These properties are intended to be exploited into scripts.

It is recommended to specify a kind of working directory with option -path. The assigned value is accessible into scripts via the function getWorkingPath(). This working directory generally indicates the output path for copying or generating files. The developer of scripts decides how to use it.

CodeWorker interprets scripts efficiently for speed. However, it is more convenient to run a standalone executable, instead of the interpreter and some script files. Moreover, once scripts are stable, why not to compile them as an executable to run the project a few times faster? Option -c++ allows translating the leader script and all its dependencies to C++ source codes, ready-to-compile.

To facilitate the tracking of errors, an integrated debugger is called thanks to the option -debug. It runs into the console, and some classical commands allow taking the control of the execution and exploring the stack and the variables.

Here are presented all switches that are allowed on the command line:

Note that the interpreter proposes a convenient way for running a common script with arguments:

codeworker <script-file> <arg1> ... <argN> [<switch>]*

This writing replaces the more verbose:

codeworker -script <script-file> -args <arg1> ... <argN> [<switch>]*

A console mode is launched when the command line is empty. The console only accepts scripts written in the common syntax, with common functions and procedures. So, parsing and generation scripts aren't typed directly on the console.

A script in CodeWorker consists of a series of statements that are organized into blocks (also known as compound statements). A statement is an instruction the interpreter has to execute.

A single statement must close with a semicolon (';'). A compound statement is defined by enclosing instructions between braces ('{}'). A block can be used everywhere you can use a single statement and must never end with a semicolon after the trailing brace.

Comments are indicated either by surrounding the text with '/*' and '*/' or by preceding the rest of the line to ignore with a double slash ('//').

It exists three families of scripts here. To facilitate their syntax highlighting in editors, or to indicate briefly the type of the script, we suggest to employ some file extensions, depending on the nature of the script. The next table exposes the different extensions used commonly in CodeWorker.

The condition is a predicat, or its negation if the '!' symbol preceeds the predicat.

Only one predicat is accepted for the moment:

• existFunction( function-name ): the condition occurs if the function named function-name already exists in the scope.

Example:

#if existFunction(f)
function g() {
  return "result of f(): " + f();
}
#else
function f() {
  return "f() was missing: declare it now";
}
#end


A package is an extension of the scripting language that allows adding new functions in CodeWorker at runtime. A package is implemented as an executable module, which exports all new functions the developer wants to make available in the interpreter.

Loading of a package

The preprocessor directive #use tells the interpreter that it must extend itself with the functions exposed by a package.

The syntax is: #use package-name

Loading a package more than once has no effect.

The name of the package must prefix the name of the function, when calling it: package-name::my-function(parameters...)

Example:

#use PGSQL
PGSQL::connect("-U pilot -d emergencyDB");
local sRequest = "SELECT solution FROM average_adjustment WHERE damage = 'broken wing'";
local listOfSolutions;
PGSQL::selectList(sRequest, listOfSolutions);
if listOfSolutions.empty()
  traceLine("No solution. Suggestion: parachute jump?");
else {
  traceLine("Solutions:");
  foreach i in listOfSolutions
    traceLine(" -" + i);
}
PGSQL::disconnect(); // if the plane hasn't crashed yet


The PGSQL package serves here for connecting to and querying a PostGreSQL database. For this example, the package exports three functions: PGSQL::connect, PGSQL::selectList and PGSQL::disconnect.

The executable module

CodeWorker expects a dynamic library, whose name is deduced from the package name and from the platform the interpreter is running to.
The short name of the dynamic library concatenates "cw" at the end of the package name. The extension of the dynamic library must be ".dll" under Microsoft Windows, and ".so" under Linux.

You must put the dynamic library at a place where CodeWorker will find it at runtime.
Microsoft Windows proceeds in the following order to locate the library:

• The directory where the executable module for the current process is located.
• The current directory.
• The Windows system directory (not recommended - it concerns CodeWorker only).
• The Windows directory (not recommended - same reason).
• The directories listed in the PATH environment variable.

So, when CodeWorker reads #use PGSQL, it searches a dynamic library called "PGSQLcw.dll" under Windows or "PGSQLcw.so" under Linux.

Building a package

This section is intended to those that want to build their own packages, for binding to a database or to a graphical library ... or just for gluing with their own libraries.

When the interpreter find the preprocessor directive #use package-name in a script, it loads the executable module and executes the exported C-like function CW4DL_EXPORT_SYMBOL void package-name_Init(CW4dl::Interpreter*).

The preprocessor definition CW4DL_EXPORT_SYMBOL and the namespace CW4dl are both declared in the C++ header file "CW4dl.h". This header file is located in the "include" directory if you downloaded binaries, and at the root of the project if you downloaded sources.

The C-like function 'package-name_Init()' MUST be present! C-like means that it is declared extern "C" (done by CW4DL_EXPORT_SYMBOL).

Initializing the module that way is useful for registering new functions in the engine, via the function createCommand() of the interpreter (see the header file "CW4dl.h" in the declaration of the class Interpreter for learning more about it).

Every function to export must start its declaration with the preprocessor definition CW4DL_EXPORT_SYMBOL (means 'extern "C"', but a little more under Windows).

• Up to 4 parameters, the signature of such a function looks like:
  

  CW4DL_EXPORT_SYMBOL const char*
 selectList(CW4dl::Interpreter*,
        CW4dl::Parameter p1, CW4dl::Parameter p2);


  where selectList is a function expecting 2 parameters.

  The initializer PGSQL_Init() in our example informs the engine about the existence of this function selectList in the package:
  

  createCommand("selectList", VALUE_PARAMETER, NODE_PARAMETER);

  
  which means that selectList expects a string followed by a tree.

  In the body of the function 'selectList(...)', the C++ binding is obtained easily by a cast of CW4dl::Parameter:

  • (const char*) p1 for the value parameter p1,
  • (CW4dl::Tree*) p2 for the node parameter p2,
• if a function contains strictly more than 4 parameter, its signature changes and requires a variable number of parameters:

  CW4DL_EXPORT_SYMBOL const char*
 myFunction(CW4dl::Interpreter*,
        int nbParams, CW4dl::Parameter* tParams);


  where tParams is an array of parameter types, and where 'nbParams' gives the size.

  The initializer PGSQL_Init() informs the engine about the existence of this function in the package differently too:

  createCommand("myFunction", 6, tParams);

  
  which means that myFunction has 6 parameters whose types are provided in tParams.

Every function returns const char*. The CodeWorker's keyword null designates an atypical tree node. It doesn't accept navigation and reference, only passing by parameter to a function. On the C++ side, this null tree node is seen as a null pointer of kind CW4dl::Tree*.

The interpreter CW4dl::Interpreter represents the runtime context of CodeWorker. It is the unavoidable intermediary between the module you are building and CodeWorker.
Use it for:

• registering new functions into the CodeWorker's engine,
• throwing an error,
• handling parse trees,

The directives #reference and #attach serve to be notified when a change has been made into a script for generating in a given language, but not taken back in another language. For example, you are writing a framework both in C++ and JAVA. You are adding some new features in C++ or correcting some mistakes. One day, you'll be care not to forget to update the JAVA generation. In fact, thanks to these directives, a warning will be produced up to changes will have been put in the other script.

How does it work? Directives must delimit the piece of script you have changed:
"#reference" key
...
"#end" key

The key is an identifier that allows putting more than one reference area into a script file. A #reference area might cover one or more #reference directives, without confusing about boundaries. The directive must be put at the beginning of the line.

Here are the directives delimiting the piece of script that should be updated later in another file:
"#attach" reference-file ':' reference-key
...
"#end" reference-key

A #attach area might cover one or more #reference or #attach directives, as a #reference area. The directive must be put at the beginning of the line.

The first time CodeWorker will encounter the reference script file, it will compute a number that depends on the content of the area. The first time CodeWorker will encounter an attached script file, it will get back the magic number of the reference area, found both by the file name and the key of the reference. And then, at the beginning, the reference and attached areas are considered as similar. CodeWorker stores the magic number of the reference just behind the #attach directive:
"#attach" reference-file ':' reference-key ',' reference-number

increment(index)

• Floating-point numbers are represented as they are commonly admitted into programming languages: 3.141592 or 5.5E+6.
• Integers are represented without the dot, 64 for instance.
• A character literal is represented between single quotes as in C or JAVA; it admits classical escape characters.
• Bytes are represented as a couple of hexadecimal digits. The 4D byte is the ASCII of the letter N.
• About boolean types, an empty string "" means false, and any kind of sequence of characters means true, such as "1" or "raspberries". Two constant literals are provided: keyword true is worth "true" and false is an empty string.
• dates are written according to a format that looks like 24sep2002, where:
  • day takes 2 digits
  • month is represented as the 3 first letters of the corresponding english word ; aug as august and may as may, for instance
  • year takes 4 digits: 2002 but never 02.
• the time representation conforms to the format:
  HH:MM:SS.millis

A constant tree describes a tree as a list of constant trees and expressions, intended to be assigned to a variable. Example:

local aVariable = "a"{["yellow", "red":"or"{.alternative="orange"

The next table exposes all pre-defined variable names (accessible from anywhere) and their meaning:

A variable that is read without being declared returns an empty string, but doesn't cause the creation of a sub-node. The danger is that you aren't safe from a spelling mistake. To prevent it, put the option -varexist on the command line and use the function existVariable() to check whether a variable exists or not.

To declare a variable to the stack, use the following declaration statement:
local-variable-statement ::= "local" local-variable-declaration ';'
local-variable-declaration ::= variable [ '=' assignment-expression ]?
assignment-expression ::= constant-tree | expression
constant-tree ::= [tree-value]? '{' [tree-array-or-attribute [',' tree-array-or-attribute]* ]? '}'
tree-value ::= expression
tree-array-or-attribute ::= tree-array | tree-attribute
tree-attribute ::= '.' attribute-name '=' assignment-expression
tree-array ::= '[' tree-array-item [',' tree-array-item]* ']'
tree-array-item ::= expression ':' assignment-expression | assignment-expression

An extension of the syntax allows the declaration of more than one variable in one shot. A comma separates the variable declarations:
local-variable-statement ::= "local" local-variable-declaration [ ',' local-variable-declaration ]* ';'

The local variable points to a new empty tree, pushed into the stack.

• If an expression is present after the local declaration, it is evaluated and the string result is assigned to the new local variable.
• If a constant tree is present after the local declaration, it is assigned to the new local variable. Example:
  

  local aVariable = {"a", {"yellow", "red"}, "submarine"};

  
  is equivalent to:
  

  local aVariable;
pushItem aVariable = "a";
pushItem aVariable;
pushItem aVariable#back = "yellow";
pushItem aVariable#back = "red";
pushItem aVariable = "submarine";


  
  where pushItem means that a new item has to be added in the array owned by aVariable, and where #last means accessing to the last item of the array.

To assign a reference to another variable, instead of either the result of evaluating an expression or a constant tree, use rather the following declaration statement:
local-ref-statement ::= "localref" local-ref-declaration [ ',' local-ref-declaration ]* ';'
local-ref-declaration ::= variable '=' reference

In the case of a CodeWorker version strictly older than 1.13, local variables that are declared in the body of a script or in the scope of a function may be accessed further in the scope of functions during their timelife. So a different behaviour may occur with a more recent CodeWorker interpreter.

This stack management had historical reasons, but it is now obsolete and often reflects an implementation's error. To preserve you from this kind of mistake, a warning may be displayed, so that scripts strictly older than version 1.13 may continue to run. Specify a version strictly older than 1.13 to the command line (option -version) for reclaiming that CodeWorker checks and generates a warning.

To correct this kind of mistake in old scripts, the variable should be propagated in an argument for functions that refer to it.

To declare a global variable, use the global statement. The declaration of a global variable can be specified anywhere in scripts. The first time the declaration of a global variable is encountered, the interpreter registers it as accessible from any point into scripts. The second time the interpreter encounters a global declaration for the variable, the latter remains global but its content is cleared.
Note that if a local variable or an attribute of the current node (this) is identical to the name of an existing global variable, the global variable remains hidden while the flow of control hasn't left the scope that contains the homonym.

the global declaration statement looks like:
global-variable-statement ::= "global" global-variable-declaration [ ',' global-variable-declaration ]* ';'
global-variable-declaration ::= variable [ '=' assignment-expression ]?

If the branch isn't known before runtime, it may be build during the execution.

Example: while parsing an XML file, each time an XML attribute is encountered, one creates the corresponding attribute into the parse tree. But the name of the attribute is discovered during the parsing. The directive #evaluateVariable(expression) allows doing it. expression is evaluated at runtime and provides a branch:

#evaluateVariable("a.b.c")

A node may contain an array of nodes, which are indexed by a key that is a constant string. A branch allows navigating through arrays, and the definitive syntax of branches conforms to:
branch ::= "#evaluateVariable" '(' expression ')'
                ::= variable ['.' sub-node | array-access]*
array-access ::= '[' expression ']'
                ::= '#' ["front" | "back" | "parent"] | "root"]
                ::= '#' '[' integer-expression ']'

20.4.1 Presentation