Basic Syntax

Optional element to specify a target platform and version of the DimML code that is used. When present this should be the first element in a DimML file. Though all platform servers can send requests to all connected servers, adding a version immediately points the file to the right server.

Example

version '2.2-adversitement'

The import statement is used to import other DimML files. DimML borrows ideas from aspect-oriented programming to support the separation of these aspects into distinct DimML files.

Whenever the DimML platform receives a request from a webpage, an attempt is made to discover DimML files that are relevant to the page. The default behavior is to use the domain of the request URL to create a list of relevant DimML files. For instance, assuming the domain is www.example.com the list would be:

• example.com.dimml
• .www.example.com.dimml
• .example.com.dimml
• .com.dimml

Any missing files are skipped. The remaining files can include import directives that will add additional DimML files to the end of the list like this

import 'customer/settings.dimml'

The DimML platform loads and merges the files in the order specified.

Each DimML file is identified by an absolute URI.

Imports refer to other DimML files using relative or absolute URIs. If the previous import statement is a part of /prod/www.example.com.dimml, the DimML file that will also be loaded as a result of the import is  /prod/customer/settings.dimml.

Imports can only be done on a global level (not within a concept). How the content of the different DimML files is merged is described in the concept section.

The concept statement is used to declare a concept and optionally inherit definitions from other concepts. Concepts are used to group logic and perform a specific task defined in DimML code. For each execution, only 1 concept is active and executed (apart from extended concepts). To identify which concept to use, a match statement is used.

In defining data collection logic for a website, it is often the case that some part of the collection process is reused in different places. DimML offers the concept of ‘extending’ or ‘inheriting’ logic to support this reuse.

concept A extends B, C {
  ...
}

As the example demonstrates, a concept can extend more than one other concept. To do so, separate the concepts using a comma (,). When a concept A extends B, all of B’s values, flows and plugins are available to concept A as if they were declared in A. When A and B both define a value or plugin with the same name, the object in concept A is used. The objects in A ‘overwrite’ the objects in B.

Note that when extending concepts defined in other files, the path should be included in the reference to the concepts. If for instance concepts B and C are defined in a functions DimML file, the code would look like this

import 'lib/functions.dimml'

concept A extends 'lib/B', 'lib/C' {
  ...
}

URL match

Matching page URLs to a concept can be done using a combination of various ‘match‘ instructions. They are combined using an ‘or‘ operator. The asterisk (*) wildcard is used for any number of characters (similar to the file system wildcard).

concept A {
  match '*/index.html'
  match '*/welcome.html'

}

A match statement is first split into its URI components: scheme, authority, path and query. For example:

match 'http://example.com/products/books?size=large'

The components will be:

• scheme: http
• authority: example.com
• path: /products/books
• query: ?size=large

The interpretation is that a specific URI will match this rule if and only if:

• it’s scheme equals ‘http’
• it’s authority equals ‘example.com’
• it’s path equals ‘/products/books’
• it’s query string contains at least one ‘size’ parameter with value ‘large’

Relative URIs are also allowed (eg. /products/books, //example.com, ?size=large, etc.). When a relative URI is used in the match expression, matching will only occur on the components that are present. For example:

match '/products/books'

will match all of

https://example.com/products/books#ch1

http://audacis.org/products/books?user=david

blabla://david:password@blabla/products/books

Matching is done by traversing each of the file and checking each concept and match statement. This is done in order of loading and importing the files and the sequence top to bottom of the concepts. If a match can be made, the concept is chosen as the one use for the specific request.

Query matching

Matching on the query component of a URI works differently. The query component of a match expression is split into its parameters. The interpretation is that these parameters should be present in the target URI to match, but not necessarily in the same order. When the value is omitted, the parameter is only tested for its presence and it can have any value. For example:

match '?size=large&user'

will first be split in its URI components (only a query component) and then into its query parameters:

• size = large
• user =

This will match

http://example.com/products/books?user=david&size=large

It will not match

http://example.com/index?size=small&user=david

Prematching

The special function $preMatch allows interaction with the client-side matching process. It can only be defined within the global scope of a DimML file; it has no effect when defined inside a concept. The signature of the function definition is:

def $preMatch = {doMatch => `
    ...
`}

Note that as in other DimML elements backticks (  `  ) are used for code snippets. By defaults the code snippets contain Javascript but other languages can be used as well (see val statement for more explanations). Prematching can only be done using default Javascript code snippets.

Control over when the matching process (for this particular DimML file) should start and select a concept is relinquished to the $preMatch function. The doMatch parameter is a callback function that should be executed when the matching process should select a concept. Some examples:

• Wait for 1 second before matching

def $preMatch = {doMatch => `
    setTimeout(doMatch,1000);
`}

• Wait (polling) for the availability of some javascript library

def $preMatch = {doMatch => `
    var handle = setInterval(function(){
        if (window.jQuery) {
            clearInterval(handle);
            doMatch();
        }
    },100);
`}

• Wait for the availability of some javascript you can edit

def $preMatch = {doMatch => `
    if (window.myLib) {
        doMatch();
    } else {
        window.dimml.doMatch = doMatch;
    }
`}

In the other javascript lib:

if (window.dimml && window.dimml.doMatch) window.dimml.doMatch();

• Match multiple times to select a concept based on client-side events

def $preMatch = {doMatch => `
    var handle = setInterval(function(){
        if (!window.s_c_il || !window.s_c_il[0] || !window.s_c_il[0].t)     
            return;
        var s = window.s_c_il[0];
        s.tttt = s.t;
        s.t = function(){s.tttt();doMatch();}
        clearInterval(handle);
        doMatch();
    },250);
`}

NOTE:

When you define a function with a name that starts with the dollar sign ($) it is considered to be a platform directive instead of an actual function to be used in your DimML program. This notion of platform directives by means of the dollar sign will be extended to other parts of a dimml program in the future. You should refrain from starting names with the dollar sign.

Services

DimML applications are typically consumers: they act when called upon (e.g. when a page is opened). However it is also possible to start a data stream from initiated by DimML itself. These are called DimML services. The use of services (producer) requires 3 changes to a consumer DimML application:

• The DimML file is not written for a specific domain, so any name can be used
• The extension should be changed to .service (in stead of .dimml)
• The match rule should include the schedule at which the DimML file will execute

For example the file www.o2mccron.service can be used to define a DimML service (eventhough the domain does not function as a website). The service concept runs based on the provided cron schedule:

concept CheckDB {
  match `0 0/5 * * * ?`@cron

  flow
    => sql['...', `DB`, batch = '1']
    => select[`field < 0`@groovy]
    => ...
    => mail[`MAIL_OPTIONS`]
}

This DimML application will run (once) every 5 minutes, perform a look up into the database and send an email if a certain condition is met. As you can see, the match statement is written as code snippet (between backticks). To avoid the interpretation of this code snippet as Javascript, the @cron statement is added. This is the only correct use of @cron.

An example of an extensive schedule:

concept CheckDB {
  match `0 30 10-13 ? * WED,FRI`@cron

  flow
  => ..
}

This will trigger a DimML service at 10:30, 11:30, 12:30, and 13:30, on every Wednesday and Friday.

By default all services have a limited lifetime (to prevent forgotten services that run perpetually). Services defined in the sandbox have a lifetime of 1 hour, development services have a lifetime of 1 day and production services have a lifetime of 1 year.

A special schedule is the ‘producer’ schedule:

match `keepalive`@cron

This schedule sends a pulse every few minutes to the first flow element in the default flow(s). It is used to keep ‘producer’ flows alive. A producer flow element does not require input data but is allowed to emit data down the flow whenever it is available. An example is the twitter flow

The val statement declares the collection of values (either client or server-side). The val statement is only valid inside a concept. Each val statement is executed in parallel with the other val statements; no sequence is defined.

val type = 'example'

A val statement can also be assigned code, by means of backticks. These backticks are used for other DimML elements as well (def, plugins, e.o.). The backtick can be extended with an @ to define which programming language the code to use for parsing and executing. By default client-side Javascript is used. Therefore

val url = `document.location.href`

is the same as

val url = `document.location.href`@javascript

Furthermore you can call functions (see def statement) which are defined in the DimML file as well. If for example you have defined a function in groovy to define the page name, the (server-side) val statement would look like this

val pagename = `getPageName()`@groovy

Constants are defined similarly to fields defined with the val statement. However constants are typically for internal use: they can be used in connection details or similar information, but they are not a part of the data tuple. Furthermore as expected from constants: the value cannot be changed throughout the execution of the DimML file. For example:

concept Global {
  match '*'

  const CPT = 'Global'

  val a = `"the current concept is ${CPT}"`@groovy
  val b = `CPT`@groovy

  flow
  =>json
  =>debug

  plugin debug

}

results in the output

{"a":"the current concept is Global","b":"Global"}

is the same as

val url = `document.location.href`@javascript

Note that the constant CPT is not a part of the output. To avoid confusion it is recommended to name constants in such a way that they can be clearly identified as constants (e.g. all-caps). This will avoid the case where vals overwrite consts if they have the same name. Note that naming is case sensitive (which is why capital casing will result in constants and vals begin distinguished).

Predefined constants
Every DimML application contains the constant _DEV. This constant is a boolean indicating if the application is running in a test environment (any environment that is not production) or not. This is especially usefull for writing a DimML application which for running in test or in production should take different actions. For instance the test DimML application should get files from the test folder, while the production application should use prod. The logic of this example can be combined in this code:

`_DEV?'test':'prod'`@groovy

Flows are used to declare data processing and export steps as a directed graph. Flows can only be used inside a concept. As input for the flows the values for all the variables (vals) are available.

Example:

concept all {
   match '*'
   val pagename = 'home'

   flow 
   => debug

   plugin debug

}

The result of this DimML code is the value ‘home’ written to the Javascript console. As the flow is initiated, the value of the pagename field is available. Note that the flow is executed for each request send to this DimML application.

Flows statements can also contain a name between brackets. The name between brackets identifies the bucket that functions as the source of data. The ‘default‘ bucket is a special bucket that contains the data collected through val definitions. If no bucket is specified the ‘default‘ bucket is assumed (as you can see in the first flow example). Furthermore you can point from any flow to any other flow and multiple flows. As a results DimML allows you to quickly create multiple data streams. In the example below you can see how the previously defined flow results in 2 additional flows

concept all {
   match '*'
   val pagename = 'home'

   flow 
   => code[views = '1'] (debug)
   => json (ftp)

   flow (debug)
   => debug

   flow (ftp)
   => ftp[
	host = 'localhost',
	user = 'user',
	password  = '123456789',
	dir = '/tmp/upload',
	flush = 'true'
     ]

   plugin debug

}

It is also possible to send the output of a flow to multiple destinations (with different flow names). For instance the DimML code below will format the data tuple and continue with two different flows.

   flow 
   => json (goto-1) (goto-2)

This notation is particularly usefull for the if flow element.

Parameters

Several flow elements may need parameters. These parameters contain string, regular expressions or code snippets. They should be separated with a comma (,) and can optionally be named. Note that only server side code is allowed because the flows are executed server side. By default server-side Javascript is used for the code snippets in the flow elements. If you want to use groovy, you can add @groovy after the last backtick. Parameters are placed within square brackets.

concept all {
   match '*'

   val pagename = 'home'

   flow 
   => select[`typeof(pagename) != 'undefined']
   => code[views = '1', pagenamelength = `pagename.size()`@groovy] 
   => filter['pagename','pagenamelength','views']
   => csv[mode = '1']
   => debug

   plugin debug

}

This will result in the following code in the Javascript console

home;4;1

To define a function the DimML language uses the def keyword. DimML functions can be written in both client and server side code. They can be called from value collection, flows and other functions. Functions can be inherited and overwritten using concept merging and global scope

Example:

def getQuery = `location.search`

This defines a client side function named getQuery that returns the query portion of the current URI. Writing a function definition on a single line will be interpreted as an expression. A multi-line function definition will expect a function body (e.g. in Javascript this should contain a return statement). You can use the client side function when collecting values:

val query = `getQuery()`

By default code snippets (and therefore also function definitions) are defined in Javascript. To define and execute server side functions, use the @groovy statements:

def getName = `return 'John Doe'`@groovy

...

val query = `getQuery()`@groovy

For defining a function that takes parameters, the => notation is used. Multiple parameters are separated by a comma.

val sessionId = `getCookie('session')`

def getCookie = {name => `
  var i,x,y,cookies=document.cookie.split(";");
  for (i=0;i<cookies.length;i++) {
    x=cookies[i].substr(0,cookies[i].indexOf("="));
    y=cookies[i].substr(cookies[i].indexOf("=")+1);
    x=x.replace(/^\s+|\s+$/g,"");
    if (x==name) {return unescape(y);}
  }
  return false;
`}

Client side functions can also be called from various plugins. For instance, the script plugin allows you to attach any Javascript code to a concept (this code will end up in the browser when concept is triggered). Functions are also supported there:

plugin script `
  console.log('the session cookie is: '+getCookie('session'));
`

An example of using a server side function inside a flow

val url = `location`

flow
  => select[`url.size() > 200`@groovy]
  => code[query = `getQuery(url)`@groovy]
  => json

Acting on client-side events allows fine-grained control over data collection and processing. Handling these events in DimML consists of two parts: Declaring the event using the event statement, and defining the appropriate logic when the event occurs.

event

A client-side event can be declared using the event element:

concept Global {
  match '*'

  event click = `window.click`
}

Defining an event in concept-scope will cause the client-side handler to be registered for that particular concept only (only if also an on event statement is defined, otherwise nothing is done). An event can also be defined in global scope. This will associate the event with every concept in scope:

event click = `window.click`

concept Global {
  match '*'
  ...
}

Event expressions consist of two parts. The part before the last dot (.) is the selector and after the dot is the event name. The selector defines an element on the page. The event name defines at what interaction on the element a DimML event should be triggered. In the example above the selector is ‘window’ and the event name ‘click’. Some more examples:

event buttonClick = `document.getElementById('button').click`
event overLink = `document.getElementsByTagName('a').mouseover`
event exitLink = `$('a.ad').click`

The selector can be a regular HTML DOM element, a list of DOM elements or a jQuery selector.

Furthermore an event definition can be extended with a function. The event will then only trigger when the expression evaluates to ‘Javascript truthy‘. Inside the function, the element that triggered the event can be accessed using ‘it‘ as a reference. In the example below you can find DimML code which sends a link’s destination URL to the Javascript console if you move your mouse of the link.

event overLink = `document.getElementsByTagName('a').mouseover`
              => `{destination:it.target}`

concept Global {
  match '*'

  on overLink include {
    flow => debug
  }

  plugin debug
}

on … do / include

As shown in the last example, the ‘on‘ keyword is used to specify an action when a DimML event occurs. When ‘include‘ follows the name of the event, any logic defined in the enclosing concept is used as well. Alternatively ‘do‘ can be used instead of ‘include‘, which omits the details of the parent concept.

Therefore this DimML code

event overLink = `document.getElementsByTagName('a').mouseover`
               => `{destination:it.target}`

concept Global {
  match '*'

  val main = 'true'

  on overLink do CollectLink
}

concept CollectLink {
  val link = 'true'

  flow => debug

  plugin debug
}

Results in

{link=true, destination=http://www.dimml.io/downloads.html}

While using the exact same code and only changing do CollectLink to include CollectLink will result in

{main=true, link=true, destination=http://www.dimml.io/downloads.html}

Plugins are used to extend the basic features of DimML. They are triggered when certain events occur, like building client side Javascript code, sending data to a flow, etc. Through this event mechanism a plugin can influence the way DimML responds to requests or trigger specific requests to get data for a specific plugin. Only one plugin of a specific type can be active for a concept. For instance, the script plugin allows the insertion of arbitrary Javascript code in a concept.

concept A {
  plugin script `console.log('A')`
}
concept B {
  plugin script `console.log('B')`
}
concept Global extends A,B {
  match '*'
}

This will result in the text ‘A’ written to the Javascript console (and not ‘B’) since only 1 script plugin can be active.

More details on the exact working of the plugins can be found on the plugins page.

A big part of the DimML language is the use of code snippets written in non DimML code. The snippets are enclosed by backticks. DimML can interpret these snippets written in different programming languages, currently Groovy, client side Javascript and server side Javascript. The default language differs by language element:

• val, const, def: client side Javascript
• flow elements, plugins: groovy

This means that

val url = `document.location.href`

will run in client side javascript. While

flow
=> code [page=`url`]

will run in groovy.

The default language can be overwritten for a specific code snippets by adding either @groovy, @javascript or @dimml for Groovy, client side Javascript and server side Javascript respectively after the closing backtick of the code snippet. Note that not all overwrites are possible; if the execution takes place on the server for instance, client side scripting is no longer possible.

For example

val url = `getURL()`@groovy

will define a val which gets its value using Groovy exection.

The default language can also be changed between statements in a dimml file or concept. The new default language is set from that point forward until the scope ends. The code below illustrates that.

@groovy

def fun1 = `...` // groovy

concept A {
  val v1 = `...` // groovy

  @javascript
  val v2 = `...` // javascript
  def fun2 = `...` // javascript
}

def fun3 = `...` // groovy