The scripting language

CodeWorker must be seen as a script interpreter that is intended to parse and to generate any kind of text or source code. This interpreter admits some options on the command line. Some of them look like those of a compiler.

CodeWorker doesn't provide any Graphical User Interface, but a console mode allows interactivity with the user.

• the script describes all the processing tasks for parsing text, decorating the graph and generating code ; the option of the command line is -script to execute the script,
• the script describes an extended BNF grammar ; the option of the command line is -parseBNF for executing the script and parsing the source file,
• the script describes how to generate code ; the option of the command line is -generate to execute the script and to generate the output file,
• the script describes how to expand a file ; the option of the command line is -expand to execute the script and to expand the output file into its markups,
• a file contains embedded scripts driving their own expansion ; the option of the command line is -autoexpand to execute embedded scripts located below each markups, expanding the output file on markups,
• the script describes a source-to-source translation ; the option of the command line is -translate to execute the script and to translate the source file to the output file,

To find easier a file to open for reading among some directories, the option -I specifies a path to explore. It gives more flexibility in sharing input files (both scripts and user files, excepting generated or expanded files) between directories, and it avoids relative or absolute paths into scripts.

It is possible to define some properties on the command line, thanks to option -define (or -D). These properties are intended to be exploited into scripts.

It is recommended to specify a kind of working directory with option -path. The assigned value is accessible into scripts via the function getWorkingPath(). This working directory generally indicates the output path for copying or generating files. The developer of scripts decides how to use it.

CodeWorker interprets scripts efficiently for speed. However, it is more convenient to run a standalone executable, instead of the interpreter and some script files. Moreover, once scripts are stable, why not to compile them as an executable to run the project a few times faster? Option -c++ allows translating the leader script and all its dependencies to C++ source codes, ready-to-compile.

To facilitate the tracking of errors, an integrated debugger is called thanks to the option -debug. It runs into the console, and some classical commands allow taking the control of the execution and exploring the stack and the variables.

Here are presented all switches that are allowed on the command line:

Note that the interpreter proposes a convenient way for running a common script with arguments:

codeworker <script-file> <arg1> ... <argN> [<switch>]*

This writing replaces the more verbose:

codeworker -script <script-file> -args <arg1> ... <argN> [<switch>]*

A console mode is launched when the command line is empty. The console only accepts scripts written in the common syntax, with common functions and procedures. So, parsing and generation scripts aren't typed directly on the console.

A script in CodeWorker consists of a series of statements that are organized into blocks (also known as compound statements). A statement is an instruction the interpreter has to execute.

A single statement must close with a semicolon (';'). A compound statement is defined by enclosing instructions between braces ('{}'). A block can be used everywhere you can use a single statement and must never end with a semicolon after the trailing brace.

Comments are indicated either by surrounding the text with '/*' and '*/' or by preceding the rest of the line to ignore with a double slash ('//').

It exists three families of scripts here. To facilitate their syntax highlighting in editors, or to indicate briefly the type of the script, we suggest to employ some file extensions, depending on the nature of the script. The next table exposes the different extensions used commonly in CodeWorker.

A package is an extension of the scripting language that allows adding new functions in CodeWorker at runtime. A package is implemented as an executable module, which exports all new functions the developer wants to make available in the interpreter.

Loading of a package

The preprocessor directive #use tells the interpreter that it must extend itself with the functions exposed by a package.

The syntax is: #use package-name

Loading a package more than once has no effect.

The name of the package must prefix the name of the function, when calling it: package-name::my-function(parameters...)

Example:

#use PGSQL
PGSQL::connect("-U pilot -d emergencyDB");
local sRequest = "SELECT solution FROM average_adjustment WHERE damage = 'broken wing'";
local listOfSolutions;
PGSQL::selectList(sRequest, listOfSolutions);
if listOfSolutions.empty()
  traceLine("No solution. Suggestion: parachute jump?");
else {
  traceLine("Solutions:");
  foreach i in listOfSolutions
    traceLine(" -" + i);
}
PGSQL::disconnect(); // if the plane hasn't crashed yet


The PGSQL package serves here for connecting to and querying a PostGreSQL database. For this example, the package exports three functions: PGSQL::connect, PGSQL::selectList and PGSQL::disconnect.

The executable module

CodeWorker expects a dynamic library, whose name is deduced from the package name and from the platform the interpreter is running to.
The short name of the dynamic library concatenates "cw" at the end of the package name. The extension of the dynamic library must be ".dll" under Microsoft Windows, and ".so" under Linux.

You must put the dynamic library at a place where CodeWorker will find it at runtime.
Microsoft Windows proceeds in the following order to locate the library:

• The directory where the executable module for the current process is located.
• The current directory.
• The Windows system directory (not recommended - it concerns CodeWorker only).
• The Windows directory (not recommended - same reason).
• The directories listed in the PATH environment variable.

So, when CodeWorker reads #use PGSQL, it searches a dynamic library called "PGSQLcw.dll" under Windows or "PGSQLcw.so" under Linux.

Building a package

This section is intended to those that want to build their own packages, for binding to a database or to a graphical library ... or just for gluing with their own libraries.

When the interpreter find the preprocessor directive #use package-name in a script, it loads the executable module and executes the exported C-like function CW4DL_EXPORT_SYMBOL void package-name_Init(CW4dl::Interpreter*).

The preprocessor definition CW4DL_EXPORT_SYMBOL and the namespace CW4dl are both declared in the C++ header file "CW4dl.h". This header file is located in the "include" directory if you downloaded binaries, and at the root of the project if you downloaded sources.

The C-like function 'package-name_Init()' MUST be present! C-like means that it is declared extern "C" (done by CW4DL_EXPORT_SYMBOL).

Initializing the module that way is useful for registering new functions in the engine, via the function createCommand() of the interpreter (see the header file "CW4dl.h" in the declaration of the class Interpreter for learning more about it).

Every function to export must start its declaration with the preprocessor definition CW4DL_EXPORT_SYMBOL (means 'extern "C"', but a little more under Windows).

• Up to 4 parameters, the signature of such a function looks like:
  

  CW4DL_EXPORT_SYMBOL const char*
 selectList(CW4dl::Interpreter*,
        CW4dl::Parameter p1, CW4dl::Parameter p2);


  where selectList is a function expecting 2 parameters.

  The initializer PGSQL_Init() in our example informs the engine about the existence of this function selectList in the package:
  

  createCommand("selectList", VALUE_PARAMETER, NODE_PARAMETER);

  
  which means that selectList expects a string followed by a tree.

  In the body of the function 'selectList(...)', the C++ binding is obtained easily by a cast of CW4dl::Parameter:

  • (const char*) p1 for the value parameter p1,
  • (CW4dl::Tree*) p2 for the node parameter p2,
• if a function contains strictly more than 4 parameter, its signature changes and requires a variable number of parameters:

  CW4DL_EXPORT_SYMBOL const char*
 myFunction(CW4dl::Interpreter*,
        int nbParams, CW4dl::Parameter* tParams);


  where tParams is an array of parameter types, and where 'nbParams' gives the size.

  The initializer PGSQL_Init() informs the engine about the existence of this function in the package differently too:

  createCommand("myFunction", 6, tParams);

  
  which means that myFunction has 6 parameters whose types are provided in tParams.

Every function returns const char*. The CodeWorker's keyword null designates an atypical tree node. It doesn't accept navigation and reference, only passing by parameter to a function. On the C++ side, this null tree node is seen as a null pointer of kind CW4dl::Tree*.

The interpreter CW4dl::Interpreter represents the runtime context of CodeWorker. It is the unavoidable intermediary between the module you are building and CodeWorker.
Use it for:

• registering new functions into the CodeWorker's engine,
• throwing an error,
• handling parse trees,

The directives #reference and #attach serve to be notified when a change has been made into a script for generating in a given language, but not taken back in another language. For example, you are writing a framework both in C++ and JAVA. You are adding some new features in C++ or correcting some mistakes. One day, you'll be care not to forget to update the JAVA generation. In fact, thanks to these directives, a warning will be produced up to changes will have been put in the other script.

How does it work? Directives must delimit the piece of script you have changed:
"#reference" key
...
"#end" key

The key is an identifier that allows putting more than one reference area into a script file. A #reference area might cover one or more #reference directives, without confusing about boundaries. The directive must be put at the beginning of the line.

Here are the directives delimiting the piece of script that should be updated later in another file:
"#attach" reference-file ':' reference-key
...
"#end" reference-key

A #attach area might cover one or more #reference or #attach directives, as a #reference area. The directive must be put at the beginning of the line.

The first time CodeWorker will encounter the reference script file, it will compute a number that depends on the content of the area. The first time CodeWorker will encounter an attached script file, it will get back the magic number of the reference area, found both by the file name and the key of the reference. And then, at the beginning, the reference and attached areas are considered as similar. CodeWorker stores the magic number of the reference just behind the #attach directive:
"#attach" reference-file ':' reference-key ',' reference-number

increment(index)

• Floating-point numbers are represented as they are commonly admitted into programming languages: 3.141592 or 5.5E+6.
• Integers are represented without the dot, 64 for instance.
• A character literal is represented between single quotes as in C or JAVA; it admits classical escape characters.
• Bytes are represented as a couple of hexadecimal digits. The 4D byte is the ASCII of the letter N.
• About boolean types, an empty string "" means false, and any kind of sequence of characters means true, such as "1" or "raspberries". Two constant literals are provided: keyword true is worth "true" and false is an empty string.
• dates are written according to a format that looks like 24sep2002, where:
  • day takes 2 digits
  • month is represented as the 3 first letters of the corresponding english word ; aug as august and may as may, for instance
  • year takes 4 digits: 2002 but never 02.
• the time representation conforms to the format:
  HH:MM:SS.millis

A constant tree describes a tree as a list of constant trees and expressions, intended to be assigned to a variable. Example:

local aVariable = "a"{["yellow", "red":"or"{.alternative="orange"

The next table exposes all pre-defined variable names (accessible from anywhere) and their meaning:

A variable that is read without being declared returns an empty string, but doesn't cause the creation of a sub-node. The danger is that you aren't safe from a spelling mistake. To prevent it, put the option -varexist on the command line and use the function existVariable() to check whether a variable exists or not.

To declare a variable to the stack, use the following declaration statement:
local-variable-statement ::= "local" local-variable-declaration ';'
local-variable-declaration ::= variable [ '=' assignment-expression ]?
assignment-expression ::= constant-tree | expression
constant-tree ::= [tree-value]? '{' [tree-array-or-attribute [',' tree-array-or-attribute]* ]? '}'
tree-value ::= expression
tree-array-or-attribute ::= tree-array | tree-attribute
tree-attribute ::= '.' attribute-name '=' assignment-expression
tree-array ::= '[' tree-array-item [',' tree-array-item]* ']'
tree-array-item ::= expression ':' assignment-expression | assignment-expression

An extension of the syntax allows the declaration of more than one variable in one shot. A comma separates the variable declarations:
local-variable-statement ::= "local" local-variable-declaration [ ',' local-variable-declaration ]* ';'

The local variable points to a new empty tree, pushed into the stack.

• If an expression is present after the local declaration, it is evaluated and the string result is assigned to the new local variable.
• If a constant tree is present after the local declaration, it is assigned to the new local variable. Example:
  

  local aVariable = {"a", {"yellow", "red"}, "submarine"};

  
  is equivalent to:
  

  local aVariable;
pushItem aVariable = "a";
pushItem aVariable;
pushItem aVariable#back = "yellow";
pushItem aVariable#back = "red";
pushItem aVariable = "submarine";


  
  where pushItem means that a new item has to be added in the array owned by aVariable, and where #last means accessing to the last item of the array.

To assign a reference to another variable, instead of either the result of evaluating an expression or a constant tree, use rather the following declaration statement:
local-ref-statement ::= "localref" local-ref-declaration [ ',' local-ref-declaration ]* ';'
local-ref-declaration ::= variable '=' reference

In the case of a CodeWorker version strictly older than 1.13, local variables that are declared in the body of a script or in the scope of a function may be accessed further in the scope of functions during their timelife. So a different behaviour may occur with a more recent CodeWorker interpreter.

This stack management had historical reasons, but it is now obsolete and often reflects an implementation's error. To preserve you from this kind of mistake, a warning may be displayed, so that scripts strictly older than version 1.13 may continue to run. Specify a version strictly older than 1.13 to the command line (option -version) for reclaiming that CodeWorker checks and generates a warning.

To correct this kind of mistake in old scripts, the variable should be propagated in an argument for functions that refer to it.

To declare a global variable, use the global statement. The declaration of a global variable can be specified anywhere in scripts. The first time the declaration of a global variable is encountered, the interpreter registers it as accessible from any point into scripts. The second time the interpreter encounters a global declaration for the variable, the latter remains global but its content is cleared.
Note that if a local variable or an attribute of the current node (this) is identical to the name of an existing global variable, the global variable remains hidden while the flow of control hasn't left the scope that contains the homonym.

the global declaration statement looks like:
global-variable-statement ::= "global" global-variable-declaration [ ',' global-variable-declaration ]* ';'
global-variable-declaration ::= variable [ '=' assignment-expression ]?

If the branch isn't known before runtime, it may be build during the execution.

Example: while parsing an XML file, each time an XML attribute is encountered, one creates the corresponding attribute into the parse tree. But the name of the attribute is discovered during the parsing. The directive #evaluateVariable(expression) allows doing it. expression is evaluated at runtime and provides a branch:

#evaluateVariable("a.b.c")

A node may contain an array of nodes, which are indexed by a key that is a constant string. A branch allows navigating through arrays, and the definitive syntax of branches conforms to:
branch ::= "#evaluateVariable" '(' expression ')'
                ::= variable ['.' sub-node | array-access]*
array-access ::= '[' expression ']'
                ::= '#' ["front" | "back" | "parent"] | "root"]
                ::= '#' '[' integer-expression ']'

2.4.1 Presentation

arithmetic-expr ::= comparith-expr [boolean-op comparith-expr]*
comparith-expr ::= sum-expr [comparison-op sum-expr]
sum-expr ::= shift-expr [['+' | '-'] shift-expr]*
shift-expr ::= factor-expr [["<<" | ">>"] factor-expr]*
factor-expr ::= literal-expr [['*' | '/' | '%'] literal-expr]*
unary-expr ::= literal-expr ["++" | "--"]
literal-expr ::= string | variable-expr | number | unary-expr
                ::= '~' literal-expr
preprocessor-expr ::= '#' ["LINE" | "FILE"]

where:

Of course, it exists some functions to handle strings as number and to execute an arithmetic operation (the 'add()' or 'mult()' functions for instance) or a comparison (the 'isPositive()' or 'inf()' functions for instance).

However, it appears clearly more convenient to write arithmetic operations and comparisons in a natural way, using operators instead of the corresponding functions. So, CodeWorker provides an escape mode that draws its inspiration from LaTeX to express mathematical formulas: the arithmetic expression are delimited by the symbol '$'.

Example:


local a = 11;
local b = 7;
traceLine("Classical mode = '"
    + inf(add(mult(5, a), 3), sub(mult(a, a), mult(b, b))) + "'");
traceLine("Escape mode = '" + $5*a + 3 < a*a - b*b$ + "'");


Classical mode = 'true'
Escape mode = 'true'

The if statement evaluates the expression following immediately. The expression must be of arithmetic, text, variable or condition type. In both forms of the if syntax, if the expression evaluates to a nonempty string, the statement dependent on the evaluation is executed; otherwise, it is skipped.

In the if...else syntax, the second statement is executed if the result of evaluating the expression is an empty string. The else clause of an if...else statement is associated with the closest previous if statement that does not have a corresponding else statement.

The while statement lets you repeat a statement or compound statement as long as a specified expression becomes an empty string. The expression in a while statement is evaluated before the body of the loop is executed. Therefore, the body of the loop may be never executed. If expression returns an empty string, the while statement terminates and control passes to the next statement in the program. If expression is non-empty, the process is repeated. The while statement can also terminate when a break, or return statement is executed within the statement body. When a continue statement is encountered, the control breaks the flow and jumps to the evaluation of the expression.

Note that the break and continue statements apply to the first loop statement (foreach/forfile/select, do/while) they encounter while leaving instruction blocks.

The BNF representation of the do statement is:
do_statement ::= "do" statement "while" expression ';'

The do-while statement lets you repeat a statement or compound statement until a specified expression becomes an empty string. The expression in a do-while statement is evaluated after the body of the loop is executed. Therefore, the body of the loop is always executed at least once. If expression returns an empty string, the do-while statement terminates and control passes to the next statement in the program. If expression is non-empty, the process is repeated. The do-while statement can also terminate when a break, or return statement is executed within the statement body. When a continue statement is encountered, control is transferred to the evaluation of the expression.

The BNF representation of this statement is:
switch_statement ::= "switch" '(' expression ')' '{' (label_declaration)* ("default" ':' statement)? '}'
label_declaration ::= ["case" | "start"] constant_string ':' statement

The switch statement allows selection among multiple sections of code, depending on the value of an expression. The expression enclosed in parentheses, the controlling expression, must be of string type.

The switch statement causes an unconditional jump to, into, or past the statement that is the switch body, depending on the value of the controlling expression, the constant string values of the case or start labels, and the presence or absence of a default label. The switch body is normally a compound statement (although this is not a syntactic requirement). Usually, some of the statements in the switch body are labeled with case labels or with start labels or with the default label. The default label can appear only once.

The constant-string in the case label is compared for equality with the controlling expression. The constant-string in the start label is compared for equality with the first characters of the controlling expression. In a given switch statement, no two constant strings in start or case statements can evaluate to the same value.

The switch statement behaviour depends on how the controlling expression matches with labels. If a case label exactly matches with the controlling expression, control is transferred to the statement following that label. If failed, start labels are iterated into the lexicographical order, and the control is transferred to the statement following the first label that matches with the beginning of the controlling expression. If failed, control is transferred to the default statement or, if not present, an error is thrown.

A switch statement can be nested. In such cases, case or start or default labels associate with the most deeply nested switch statements that enclose them.

Control is not impeded by case or start or default labels. To stop execution at the end of a part of the compound statement, insert a break statement. This transfers control to the statement after the switch statement.

A foreach statement iterates all items of the list owned by node list-node. The iterator refers to the current item of the list, and the body statement is executed on it.

Items are iterated either in the order of entrance, or in alphabetical order if option sorted is set. The sort operates on keys, except if the option by_value is set. The order is inverted if option reverse was chosen. To ignore the case, these options must be followed by no_case. If not, uppercase letters are considered as smaller than any lowercase letter.

      // file "Documentation/ForeachSampleSorted.cws":
      local list;
      insert list["silverware"] = "tea spoon";
      insert list["Mountain"] = "Everest";
      insert list["SilverWare"] = "Tea Spoon";
      insert list["Boat"] = "Titanic";
      insert list["acrobat"] = "Circus";
     
      traceLine("Sorted list in a classical order:");
      foreach i in sorted list {
          traceLine("\t" + key(i));
      }
      traceLine("Note that uppercases are listed before lowercases." + endl());
     
      traceLine("Sorted list where the case is ignored:");
      foreach i in sorted no_case list {
          traceLine("\t" + key(i));
      }
     
      traceLine("Reverse sorted list:");
      foreach i in reverse sorted list {
          traceLine("\t" + key(i));
      }
     
      traceLine("Reverse sorted list where the case is ignored:");
      foreach i in reverse sorted no_case list {
          traceLine("\t" + key(i));
      }

Output:

Sorted list in a classical order:
    Boat
    Mountain
    SilverWare
    acrobat
    silverware
Note that uppercases are listed before lowercases.

Sorted list where the case is ignored:
    acrobat
    Boat
    Mountain
    SilverWare
    silverware
Reverse sorted list:
    silverware
    acrobat
    SilverWare
    Mountain
    Boat
Reverse sorted list where the case is ignored:
    silverware
    SilverWare
    Mountain
    Boat
    acrobat

Control may not be sequential into the body statement. break and return enable exiting definitely the loop, and continue transfers the control to the head of the foreach statement for the next iteration.

Option cascading allows propagating foreach on item nodes. The way it works is illustrated by an example:


    foreach i in cascading myObjectModeling.packages ...


Option cascading avoids writing the following code:


function propagateOnPackages(myPackage : node) {
    foreach i in myPackage {
        // my code to apply on this package
        if existVariable(myPackages.packages)
            propagateOnPackages(myPackages.packages);
    }
}
propagateOnPackages(myObjectModeling.packages);


• first means that the item is cascaded before running the body,

        // file "Documentation/ForeachSampleFirst.cws":
      local myObjectModeling;
      insert myObjectModeling.packages["Massif"] = "...";
      local myPackage;
      ref myPackage = myObjectModeling.packages["Massif"];
      insert myPackage.packages["Alps"] = "...";
      insert myPackage.packages["Himalaya"] = "...";
      insert myPackage.packages["Rock Mountains"] = "...";
      insert myObjectModeling.packages["Silverware"] = "...";
      ref myPackage = myObjectModeling.packages["Silverware"];
      insert myPackage.packages["Spoon"] = "...";
      insert myPackage.packages["Fork"] = "...";
      insert myPackage.packages["Knife"] = "...";
     
      foreach i in cascading first myObjectModeling.packages {
          traceLine("\t" + key(i));
      }

  Output:

      Alps
    Himalaya
    Rock Mountains
    Massif
    Spoon
    Fork
    Knife
    Silverware

• last is the default behaviour, as seen in previous examples, and

        // file "Documentation/ForeachSampleLast.cws":
      local myObjectModeling;
      insert myObjectModeling.packages["Massif"] = "...";
      local myPackage;
      ref myPackage = myObjectModeling.packages["Massif"];
      insert myPackage.packages["Alps"] = "...";
      insert myPackage.packages["Himalaya"] = "...";
      insert myPackage.packages["Rock Mountains"] = "...";
      insert myObjectModeling.packages["Silverware"] = "...";
      ref myPackage = myObjectModeling.packages["Silverware"];
      insert myPackage.packages["Spoon"] = "...";
      insert myPackage.packages["Fork"] = "...";
      insert myPackage.packages["Knife"] = "...";
     
      foreach i in cascading last myObjectModeling.packages {
          traceLine("\t" + key(i));
      }

  Output:

      Massif
    Alps
    Himalaya
    Rock Mountains
    Silverware
    Spoon
    Fork
    Knife

  propagates the foreach on the current item after executing the body.

A forfile statement iterates the name of all files that verify the filter file-pattern. The iterator refers to the current item of the list composed of retained file names, and the body statement is executed on it. Note that the file pattern may begin with a path, which cannot contain jocker characters ('*' and '?').

Like for the foreach statement, items are iterated either in the order of entrance, or in alphabetical order of keys if option sorted is set. To ignore the case, the option must be followed by no_case. If not, uppercase letters are considered as smaller than any lowercase letter.

Control may not be sequential into the body statement. break and return enable exiting definitely the loop, and continue transfers the control to the head of the forfile statement for the next iteration.

The option cascading allows propagating forfile on directories recursively. The way it works is illustrated by an example:

      // file "Documentation/ForfileSample.cws":
      local iIndex = 0;
      forfile i in cascading "*.html" {
          if $findString(i, "manual_") < 0$ &&
              $findString(i, "Bugs") < 0$ {
                  traceLine(i);
          }
          // if too long, stop the iteration
          if $iIndex > 15$ break;
          increment(iIndex);
      }

Output:

cs/DOTNET.html
cs/tests/data/MatchingTest/example.csv.html
Documentation/LastChanges.html
java/JAVAAPI.html
java/data/MatchingTest/example.csv.html
Scripts/Tutorial/GettingStarted/defaultDocumentation.html
WebSite/AllDownloads.html
WebSite/examples/basicInformation.html
WebSite/highlighting/basicInformation.html
WebSite/repository/highlighting.html
WebSite/repository/JEdit/Entity.java.cwt.html
WebSite/serewin/ExempleIllustre.html
WebSite/tutorials/DesignSpecificModeling/tutorial.html
WebSite/tutorials/DesignSpecificModeling/highlighting/demo.cws.html
WebSite/tutorials/overview/tinyDSL_spec.html
WebSite/tutorials/overview/scripts2HTML/CodeWorker_grammar.html

At the beginning, i points to the first HTML file of the current directory and the body is executed. Before iterating i to the next item, the forfile checks whether the directory of the current file owns subfolders or not. If yes, it applies recursively forfile on subfolders.

Option cascading offers two behaviours:

• first means that the subfolders are visited before running the body,
• last is the default behaviour, as seen in previous examples, and propagates the forfile on the subfolder after executing the body.

A select statement iterates a list of nodes that match a motif expression. The iterator refers to the current item of the list composed of retained nodes, and the body statement is executed on it.

      // file "Documentation/SelectSample.cws":
      local a;
      pushItem a.b;
      pushItem a.b#back.c = "01";
      pushItem a.b#back.c = "02";
      pushItem a.b#back.c = "03";
      pushItem a.b;
      pushItem a.b#back.c = "11";
      pushItem a.b#back.c = "12";
      pushItem a.b#back.c = "13";
      pushItem a.b;
      pushItem a.b#back.c = "21";
      pushItem a.b#back.c = "22";
      pushItem a.b#back.c = "23";
      select i in a.b[].c[] {
          traceLine("i = "+ i);
      }

Output:

i = 01
i = 02
i = 03
i = 11
i = 12
i = 13
i = 21
i = 22
i = 23

Like for the foreach statement, items are iterated either in the order of entrance, or according to the sorting result if the option sorted is set.

Control may not be sequential into the body statement. break and return enable exiting definitely the loop, and continue transfers the control to the head of the select statement for the next iteration.

Error handling is implemented by using the try, catch, and error keyword. With error handling, your program can communicate unexpected events to a higher execution context that is better able to recover from such abnormal events. These errors are handled by code that is outside the normal flow of control.

The compound statement after the try clause is the guarded section of code. An error is thrown (or raised) when command error(message-text) is called or when CodeWorker encounters an internal error. The compound statement after the catch clause is the error handler, and catches (handles) the error thrown. The catch clause statement indicates the name of the variable that must receive the error message.

A exit statement leaves the application and returns an error code, given by the integer-expression.

Example:

exit -1;

Arguments passed by parameter must be chosen among the following modes:

• value: if the mode of argument is omitted, this is the default mode ; it requires a sequence of characters (a value of node, a constant string or the result of a expression),
• node: a node is passed and it may be changed or inspected in the body. The scope of a reference assignment is limited to the scope of the function: once the function is left, the variable receives the value of the referenced node. It is explained by the fact that the parameter is a new local variable, which refers to the node passed as argument. So, a reference assignment is applied on the local variable only.
• iterator: the iterator of a foreach statement is expected, for applying iterator functions on the argument (first() for instance). Not really useful and node is now sufficient.
• reference: a node is passed and it may be changed or expected in the body. On the contrary of variable mode, a reference assignment is propagated outside the scope of the function.

If you have omitted to return a value from a function, it returns an empty string ; in that case, you expects to call this function as a procedure and the result isn't exploited. The special procedure nop takes a function call as parameter and allows executing the function and ignoring the result. It isn't compulsory to use nop for calling a function as a procedure. As in C or C++, you can type the function call followed by a semi-colon and the result is lost.

It exists two possibilities for returning a value:

• to populate an internal local variable whose name is the same as the function name,
• to use the return statement, followed by the expression to evaluate,

If you wish to execute a particular process in any case before leaving a function and:

• it exists more than one controlling sequence to leave,
• some errors may be raised,

the statement finally warrants you that the block of instructions that follows the keyword will be systematically executed before leaving. This declaration may be placed anywhere into the body of the function. Its syntax conforms to:
finally-statement ::= "finally" compound-statement

Example:

      // file "Documentation/FinallySample.cws":
      1 function f(v : value) {
      2     traceLine("BEGIN f(v)");
      3     finally {
      4         traceLine("END f(v)");
      5     }
      6     // the body of the function, with more than
      7     // one way to exit the function, for example:
      8     if !v return "empty";
      9     if v == "1" return "first";
      10     if v == "2" return "second";
      11     if v == "3" return "third";
      12     return "other";
      13 }
      14
      15 traceLine("...f(1) has been executed and returned '" + f(1) + "'");

line 3: the finally statement is put anywhere in the body,
line 4: this statement will be executed while exiting the function, even if an exception was raised,

Output:

BEGIN f(v)
END f(v)
...f(1) has been executed and returned 'first'

It may arrive that a function prototype must be declared before being implemented, because of a cross-reference with another function for instance. The scripting language offers the forward declaration to answer this need. To do that, the prototype of the function is written, preceded by the declare keyword:
forward-declaration ::= "declare" function-prototype ';'

A template function represents a set of functions with the same prototype, except the dispatching constant. The dispatching constant is a constant string that extends that name of the function. These functions instantiate the template function for a particular dispatching constant. Each instantiated function implements its own body.

The BNF representation of a template function to implement is:
template-function-definition ::= instantiated-function-definition | generic-function-definition
instantiated-function-definition ::= instantiated-function-prototype compound-statement
instantiated-function-prototype ::= "function" function-name '<' dispatching-constant '>' '(' parameters ')'
dispatching-constant ::= a constant string between double quotes
generic-function-definition ::= generic-function-prototype [compound-statement | template-based-body]
generic-function-prototype ::= "function" function-name '<' generic-key '>' '(' parameters ')'
generic-key ::= an identifier that matches any dispatching constant with no attached prototype
template-based-body ::= "{{" template-based-script "}}"
template-based-script ::= a piece of template-based script describing the generic implementation

A call to a template function requires to provide a dispatching expression to determine the dispatching constant. The dispatching expression will be evaluated during the execution and CodeWorker will resolve what instantiated function of this template to call: the result of the dispatching expression must match with the dispatching constant of the instantiated function. The BNF representation of a call to a template function is:
instantiated-function-call ::= function-name '<' dispatching-expression '>' '(' parameters ')'
parameters ::= expression [',' expression]*

Note that a dispatching constant may be empty and such an instantiated function can be called as a classical function. In fact, classical functions are considered as instantiated functions where the dispatching constant is empty.

template functions bring generic programming in the language: let imagine that we need function getType(myType : node), to decline for every language we could have to generate (C++, Java, ...). Normally, you'll write the following lines to recover the type depending on the language for which you are producing the source code:


if doc_language == "C++" {
    sType = getCppType(myParameterType);
} else if doc_language == "JAVA" {
    sType = getJAVAType(myParameterType);
} else {
    error("unrecognized language '" + doc_language + "'");
}



sType = getType<doc_language>(myParameterType);



function getType<"JAVA">(myType : node) {
    ... // implementation for returning a Java type
}

function getType<"C++">(myType : node) {
    ... // implementation for returning a C++ type
}


Trying to call an instantiated function that doesn't exist, raises an error at runtime. However, one might imagine an implementation by default. For instance:


function getType<T>(myType : node) {
    ... // common implementation for any unrecognized language
}


For those that know generic programming with C++ templates, here is a classical example of using template functions:


function f<1>() { return 1; }
function f<N>() { return $N*f<$N - 1$>()$; }
local f10 = f<10>();
if $f10 != 3628800$ error("10! should be worth 3628800");
traceLine("10! = " + f10);


10! = 3628800

To provide more flexibility in the implementation of the template function, depending on the generic key <T>, the body admits a template-based script to implement the source code of the function. The specialization of the function for a given template instantiation key is then resolved at runtime.

Example:
The template function f inserts a new attribute in a tree node. The attribute has the name passed to the generic key for instantiation, and the value of the instantiation key is assigned to the new attribute. Then, the function calls itself recursively on the instantiation key without the last character.
For instance, the source code of f<"field"> should be:

function f<"field">(x : node) {
      insert x.field = "field";
      f<"fiel">(x); // cut the last character
}


Code:

//a synonym of f<"">(x : node), terminal condition for recusive calls
function f(x : node) {/*does nothing*/}

function f<T>(x : node) {{
      // '{{' announces a template-based script, which
      // will generate the correct implementation during the instantiation
      insert x.@T@ = "@T@";
      f<"@T.rsubString(1)@">(x);
@
      // '}}' announces the end of the template-based script
}}

f<"field">(project);
traceObject(project);


Output:

Tracing variable 'project':
      field = "field"
      fiel = "fiel"
      fie = "fie"
      fi = "fi"
      f = "f"
End of variable's trace 'project'.

For more readability, syntactical facilities are offered to call functions on a node as if this function was a method of the node. For example, it is possible to call function leftString on the node a like this: a.leftString(2), instead of the classical functional form: leftString(a, 2).

The rule is that every function (user-defined included) whose first argument is passed either by value or by node or by index (but never by reference) can propose a method call.

In that case, the method call applies on the first argument, which has to be a node. The BNF representation of a method call is:
method-call ::= variable '.' function-name '(' parameters ')'
parameters ::= expression [',' expression]*
where parameters have missed the first argument of the function called function-name.

It exists some exceptions where the method doesn't apply to the first argument:

• findElement applies on the second argument,
• replaceString applies on the third argument,

• empty is a synonym as a method of the function isEmpty,
• length is a synonym for the function lengthString,
• size is a synonym for the function getArraySize,

The token filename is the argument name that the user chooses for passing the name of the file to the body of the hook.

This special function allows implementing a hook that will be called each time a read-only file will be encountered while generating the output file through the generate or expand instruction.

Limitations: only one declaration of this hook is authorized, and it can't be declared inside a parsing or pattern script.

Example:

Common usage: file to generate has to be checked out from a source code control system (see system command to run executables).

readonlyHook(sFilename) {
  if !getProperty("SSProjectFolder") || !getProperty("SSWorkingFolder") || !getProperty("SSExecutablePath") || !getProperty("SSArchiveDir") {
    traceLine("WARNING: properties 'SSProjectFolder' and 'SSWorkingFolder' and 'SSExecutablePath' and 'SSArchiveDir' should be passed to the command line for checking out read-only files from Source Safe");
  } else {
    if startString(sFilename, getProperty("SSWorkingFolder")) {
      local sourceSafe;
      insert sourceSafe.fileName = sFilename;
      generate("SourceSafe.cwt", sourceSafe, getEnv("TMP") + "/SourceSafe.bat");
      if sourceSafe.isOk {
        putEnv("SSDIR", getProperty("SSArchiveDir"));
        traceLine("checking out '" + sFilename + "' from Source Safe archive '" + getProperty("SSArchiveDir") + "'");
        local sFailed = system(getEnv("TMP") + "/SourceSafe.bat");
        if sFailed {
          traceLine("Check out failed: '" + sFailed + "'");
        }
      }
    } else {
      traceLine("Unable to check out '" + sFilename + "': working folder starting with '" + getProperty("SSWorkingFolder") + "' expected");
    }
  }
}


This special function allows implementing a hook that will be called just before writing a file, after ending a text generation process such as expanding or generating or translating text.

It is very important to notice that it returns a boolean value. A true value means that the generated text must be written into the file. A false boolean value means that the generated text doesn't have to be written into the file.

CodeWorker always interprets not returning a value explicitly of a function, as returning an empty string. If you forget to return a value, the generated text will not be written into the file!

The BNF representation of this statement is:
writefileHook-statement ::= "writefileHook" '(' filename ',' position ',' creation ')' compound-statement

Limitations: only one declaration of this hook is authorized, and it can't be declared inside a parsing or pattern script.

Example:

writefileHook(sFilename, iPosition, bCreation) {
    if bCreation {
        traceLine("Creating file '" + sFilename + "'!");
    } else {
        traceLine("Updating file '" + sFilename + "', difference at " + iPosition + "!");
    }
    return true;
}


This special function is automatically called before that the extended BNF engine resolves the production rule of a BNF non-terminal. Combined with stepoutHook(), it is very useful for trace and debug tasks.

This hook can be implemented in parse scripts only.

The BNF representation of this statement is:
stepintoHook-statement ::= "stepintoHook" '(' sClauseName ',' localScope ')' compound-statement

This special function is automatically called once the extended BNF engine has finished the resolution of a BNF non-terminal. Combined with stepintoHook(), it is very useful for trace and debug tasks.

This hook can be implemented in parse scripts only.

The BNF representation of this statement is:
stepoutHook-statement ::= "stepoutHook" '(' sClauseName ',' localScope ',' bSuccess ')' compound-statement

A statement's modifier is a directive that stands just before a statement, meaning an instruction or a compound statement.

This directive operates some actions in the scope of the statement and then restores the behaviour as being before.

This action may be:

• to measure the time that is consumed by the execution of the statement,
• to redirect into a variable all messages intended to the console during the execution of the statement,
• to push a new project parse tree,
• to change the output file during the execution of the statement, while generating text,
• to redirect the output stream into a variable during the execution of the statement, while generating text,
• to change the output file during the execution of the statement, while generating text, and to apply an expansion mode on it,

This keyword stands just before an instruction or a compound statement. It executes the statement and then, it measures the time it has consumed.

This keyword stands just before an instruction or a compound statement. It executes the statement and all messages intended to the console are concatenated into a string, instead of being displayed. The variable that receives the concatenation of messages is specified after the quiet keyword.

The BNF representation of the quiet statement modifier looks like:
quiet_modifier ::= "quiet" '(' variable ')' statement

Note that the variable must have been declared before, as a local one or as an attribute of the parse tree. If this variable doesn't exist while executing the statement, an error is raised.

This keyword stands just before an instruction or a compound statement. A new project parse tree is created, which is empty and that replaces temporarily the current one. The statement is executed and, once the controlling sequence leaves the statement, the temporary parse tree is removed, and the precedent project comes back as the current one.

The BNF representation of the new_project statement modifier looks like:
new_project_modifier ::= "new_project" statement

This statement modifier is useful to handle a task that doesn't have to interact with the main parse tree.

This keyword stands just before an instruction or a compound statement. A new standard input is opened for reading data. Generally, the keyboard is the standard input, but here, it will be the content of a file that is passed to the argument filename. Once the execution of the statement has completed, the precedent standard input comes back.

The BNF representation of the file_as_standard_input statement's modifier looks like:
file_as_standard_input_modifier ::= "file_as_standard_input" '(' filename ')' statement

This statement modifier is useful to replay a sequence of commands for the debugger or to drive the standard input from an external module that puts its instructions into a file for a batch mode or anything else.

This keyword stands just before an instruction or a compound statement. A new standard input is opened for reading data. Generally, the keyboard is the standard input, but here, it will be the content of the string that is passed to argument. Once the execution of the statement has completed, the precedent standard input comes back.

The BNF representation of the string_as_standard_input statement's modifier looks like:
string_as_standard_input_modifier ::= "string_as_standard_input" '(' expression ')' statement

The standard input is the result of evaluating expression.

This keyword stands just before an instruction or a compound statement that belongs to a parsing/translation script exclusively. A new input file is opened for source scanning, and replaces temporarily the precedent during the execution of the statement.The statement is executed and, once the controlling sequence leaves the statement, the input file is closed properly and the precedent one comes back.

The BNF representation of the parsed_file statement modifier looks like:
parsed_file_modifier ::= "parsed_file" '(' filename ')' statement

The token filename is an expression that is evaluated to give the name of the input file.

This statement modifier is useful to handle a task that must redirect the text to parse into another input file. An example could be to emulate the C++ preprocessing on #include directives.

This keyword stands just before an instruction or a compound statement that belongs to a parsing/translation script exclusively. The result of an expression is taken as the source to scan, and replaces temporarily the precedent input during the execution of the statement.The statement is executed and, once the controlling sequence leaves the statement the precedent input comes back.

The BNF representation of the parsed_string statement modifier looks like:
parsed_string_modifier ::= "parsed_string" '(' expression ')' statement

The token fexpression is an expression that is evaluated to give the text to scan.

This statement modifier is useful to handle a task that must temporary parse a string.

This keyword stands just before an instruction or a compound statement that belongs to a pattern script exclusively. A new output file is opened for source code generation, preserving protected areas as usually, and replaces temporarily the current one during the execution of the statement. The statement is executed and, once the controlling sequence leaves the statement, the output file is closed properly and the precedent one takes its place.

The BNF representation of the generated_file statement modifier looks like:
generated_file_modifier ::= "generated_file" '(' filename ')' statement

The token filename is an expression that is evaluated to give the name of the output file.

This statement modifier is useful to handle a task that must redirect the generated text into another output file. An example could be to split an HTML text to generate into a few files for implementing a frame set.

This keyword stands just before an instruction or a compound statement that belongs to a pattern script exclusively. The output stream is redirected into a variable that replaces temporarily the current output stream during the execution of the statement. The statement is executed and, once the controlling sequence leaves the statement, the variable is populated with the content of the output produced during this scope and the precedent output stream takes its place.

The BNF representation of the generated_string statement modifier looks like:
generated_string_modifier ::= "generated_string" '(' variable ')' statement

The variable argument gives the name of the variable that will be populated with the generated text. This variable must already exist, declared on the stack or referring a node of the current parse tree.

This keyword stands just before an instruction or a compound statement that belongs to a pattern script exclusively. A new output file is opened for appending source code generation at the end of the file and replaces temporarily the current one during the execution of the statement. The statement is executed and, once the controlling sequence leaves the statement, the output file is closed properly and the precedent one takes its place.

The BNF representation of the appended_file statement modifier looks like:
appended_file_modifier ::= "appended_file" '(' filename ')' statement

The token filename is an expression that is evaluated to give the name of the output file to append.

All functions and procedures that are described below may be encountered in any kind of scripts : parsing, source code generation and file expanding, process driving, included script files.

3.2 add

3.3 addGenerationTagsHandler

3.4 addToDate

3.5 appendFile

3.6 autoexpand

3.7 bytesToLong

3.8 bytesToShort

3.9 byteToChar

3.10 canonizePath

3.11 ceil

3.12 changeDirectory

3.13 changeFileTime

3.14 charAt

3.15 charToByte

3.16 charToInt

3.17 chmod

3.18 clearVariable

3.19 closeSocket

3.20 compareDate

3.21 compileToCpp

• procedure compileToCpp(scriptFileName : string, projectDirectory : string, CodeWorkerDirectory : string)

  Parameter	Type	Description
  scriptFileName	string	the name of a script file to compile to C++
  projectDirectory	string	the location where to put on the disk the scripts compiled to C++
  CodeWorkerDirectory	string	the root path of CodeWorker either in development or distributed state

  Compiles the leader script file called scriptFileName and all scripts that might be reclaimed during the execution. The corresponding C++ files are stored into projectDirectory with the makefile (a Visual C++'s DSP file). The path to libraries and the origin of some important include files is determined thanks to the path to CodeWorker put into CodeWorkerDirectory.

  The script file cannot be a pattern script or a parsing script.

  If an error occurs, an error message is raised.

  Example:

  local sScriptFile = "GettingStarted/LeaderScript6.cws";
local sDirectory = getWorkingPath() +
            "Scripts/Tutorial/GettingStarted/bin";
removeDirectory(sDirectory);
compileToCpp(sScriptFile, sDirectory, ".");
local theFiles;
if !scanFiles(theFiles, sDirectory, "", true) error("should have worked!");
traceLine("generated files:");
foreach i in sorted theFiles traceLine(" " + i);

  Output:

  generated files:
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/CGExternalHandling.h
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/CGRuntime.h
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/CppObjectBody_cwt.cpp
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/CppObjectBody_cwt.h
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/CppObjectHeader_cwt.cpp
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/CppObjectHeader_cwt.h
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/CppParsingTree.h
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/DynPackage.h
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/HTML2LaTeX_cwp.cpp
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/HTML2LaTeX_cwp.h
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/HTMLDocumentation_cwt.cpp
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/HTMLDocumentation_cwt.h
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/JAVAObject_cwt.cpp
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/JAVAObject_cwt.h
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/LeaderScript6.dsp
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/LeaderScript6_cws.cpp
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/LeaderScript6_cws.h
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/Makefile
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/SimpleML-parsing_cwp.cpp
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/SimpleML-parsing_cwp.h
    e:\Projects\generator/Scripts/Tutorial/GettingStarted/bin/UtlException.h

3.22 completeDate

3.23 completeLeftSpaces

3.24 completeRightSpaces

3.25 composeAdaLikeString

3.26 composeCLikeString

3.27 composeHTMLLikeString

3.28 composeSQLLikeString

3.29 computeMD5

• function computeMD5(text : string) : string

  Parameter	Type	Description
  text	string	the string to encrypt in MD5

  Computes the MD5 of a string.

  This optimized MD5 implementation conforms to RFC 1321.
  Source: http://www.cr0.net:8040/code/crypto/md5/
  Copyright 2001-2004 Christophe Devine

  Example:

  local sSentence = "Garfield squashed 5 spiders yesterday";
local sCode = computeMD5(sSentence);
if sCode != "B2D989F0C0501E9A9D4A9F1B4D06E2C5" {
    error("bad result from 'computeMD5()'!");
}
traceLine("computeMD5('" + sSentence + "') = " + sCode);

  Output:

  computeMD5('Garfield squashed 5 spiders yesterday') = B2D989F0C0501E9A9D4A9F1B4D06E2C5