# Task Composition

The installation and provisioning process is driven by gathering a task list from the available scopes in preparation for task execution.

## 1 Scope List

In preparation for gathering a task list, an ordered list of scopes is collected. The following process of collecting scopes is repeated once for each stream specified in the project page or in the workspace:

- Visit the product version as selected in the product page or in the installation.
  - Add that product version's containing products's containing product catalog to the list. Add that product version's containing product to the list. Add that product version itself to the list.
- Visit a stream as selected in the project page or in the workspace, if there is one.
  - Add that stream's containing projects's containing project catalog to the list. Add that stream's outer-most containing project to the list and proceed down through the nested projects up to the stream's directly containing project, adding each project to the list along the way. Add that stream itself to the list.
- Add the installation to the list.
- Add the workspace to the list.
- Add the user to the list.

For each list of scopes, an ordered list of tasks is collected. Initially, for each scope in the list, three variables are induced, one each for the name, label, and description attributes of the scope where the variable name is prefixed with the `scope type` as follows:

- `scope.product.catalog`
- `scope.product`
- `scope.product.version`
- `scope.project.catalog`
- `scope.project`
- `scope.project.stream`
- `scope.installation`
- `scope.workspace`
- `scope.user`

The value of each such variable will be the value of that attribute for the scope. If the scope's label is `null`, the name is used as the label value and if the scope's description is `null`, the scope's label is used as the description value. In addition, to the `name` variable, for each product, product version, project, and project stream an additional variable with the name suffix `.qualifier` is induced where the value is the `qualified name` of the scope. For example, the value of the `scope.project.stream.name.qualified` variable of the Oomph.setup project's master stream is `org.eclipse.oomph.master` All these induced variables are added, in scope order, to the initial gathered list of tasks.

Additional tasks are gathered into the task list from the ordered scopes by visiting each contained task of each scope as follows:

- If the task is disabled, ignore the task.
- If the list of scopes doesn't contain all of the task's restrictions, ignore the task.
- If the task is compound, recursively visit each of that compound task's contained tasks.
- Otherwise, add the task to the list of tasks. In other works, the gathered task list will only contain leaf tasks.

At the end of scope processing, the gathered task list contains all of the per-stream tasks, or just task list if there is no stream. Each task list (or the one task list) is further processed in two phases. The initial phase is applied for each task list separately. The per-stream task lists are then concatenated into a single list and the second phase is applied to that final composed list, or the one task list is further processed by the second phase. The processing of the the task list in each phase is roughly same.

## 2 Task List

The task list is processed to induce additional tasks, to override and merge tasks, to evaluate and expand variables and to reorder tasks. The members of the task list that are variables induce an initial set of keys, i.e., a set of all variable names. Oomph tasks are modeled with EMF, so each task instance knows it corresponding EMF class. During the initial phase processing, the list of tasks is analyzed to determine the set of EMF classes that are required to implement all the tasks in the list. Each EMF class is processed as follows:

- If the class contains enablement annotations, induce p2 tasks, i.e., tasks to install the necessary implementations in the installed product, and add these induced tasks to the head of the task list.
- If the class contains variable annotations, induce the variables and add them to the head of the task list.
- Visit each attribute of the class as follows:
  If the attribute's type isn't String, ignore it.
  - If the attribute has both a variable annotation and a variable rule annotation, visit each instance of that attribute's class in the task list as follows:
    - If the attribute's value in that instance task is not empty, the rule isn't needed, so ignore it.
    - If the instance task doesn't have an ID, or it does, but the key composed from the ID value, with '.' and the attribute name appended, is already in the list of induced keys, the rule isn't needed, so ignore it.
    - Induce a rule variable for that attribute, if that rule variable hasn't already been induced for another task instance.
    A mapping from rule variables to their inducing attribute is maintained.

At the end of this processing for the first phase, the list of tasks includes additional induced variables and tasks. For the final phase, the composed list of tasks contains no variables, because they've all been evaluated and expanded, and the induced tasks are already present.

Further processing proceeds as follows:

- Build a substitution map, i.e., task-to-task map of substitutions by visiting each task as follows:
  - Put the task's override token into an token-to-task map, and if another task for that token is already present in the token-to-task map, the visited task is a substitution for that other task so add that to the substitution map. As such, tasks later in the list can override tasks earlier in the list.
  The substitution map is then further processed to follow the substitution mappings so as to represent the direct substitutions.

### 2.1 Initial Phase

For the initial phase processing, all the tasks are efficiently copied, including the copying of the containing scopes. The copying process takes the task-to-task substitution map into account, i.e., each task is logically replaced in the copy by its merged override. As such, only the final overridden merged task remains in the resulting task list copy and all references to the overridden and overriding tasks will reference the final merged override instead. Further processing of the task list proceeds with this copied task list.

An explicit key map, i.e., a map from variable name to variable, is computed by visiting each variable in the task list. Note that the preceding copying process will have eliminated duplicate variables. The initial phase processing then proceeds by visiting each task with a non-empty ID attribute as follows:

- Visit each attribute of the task's EMF class with respect to the variable name induced by appending '.' and the attribute name to the value of the task's ID attribute:
  - If the attribute is the ID attribute of a task, or the attribute isn't a single-valued String-typed attribute, ignore it.
  - If the explicit keys contains the induced variable's name, the task's value for the attribute is empty, and the attribute has a variable annotation, change the task's value for the attribute to be a reference to the explicit variable.
  - If the explicit keys do not include the induced variable name, induce a new variable record that it's an explicit key, add it to the task list, and also do the following:
    - If the task's value for the attribute is empty, and the attribute has a variable annotation, change the task's value for the attribute to be a reference to the induced variable.
    - Conversely, if the task's value for the attribute is non-empty, set the induced variable's value to that value, and as for the empty value case, if the attribute has a variable annotation, change the task's value for the attribute to be a reference to the induced variable.
    - Also induce rule variables for each of the attribute's rule annotations, recording them as explicit keys.
    - And finally, if the attribute has a variable annotation, and the induced variable's value is a self reference that would lead to circular evaluation, induce a yet another variable with the induced variable's name suffixed with `.explicit` from the explicit annotations of the attribute, and also change the self-referencing variable's value to refer to that explicit variable.
- Visit each task to handle its so called active annotations. Such annotations can be used on variables to compose a set of choices from choices defined in other variables, or to induce choices from the value of another variable. TODO examples
- Visit each variable task to build a key-to-value map. Analyze that map to determine the variable references, reordering the map base on this dependency analysis. And expand the variables in the ordered map.
- Expand the values of all attributes, of all tasks based on the key-to-value map, except those attributes marked as not expandable. E.g., the name attribute of a variable itself is not expanded.
- Recall that the gathering of tasks effectively ignore compound tasks. But those compound tasks can specify predecessors and successors and well as restrictions. As such, those predecessors and successors are expanded to become references to the leaf tasks. Furthermore, the resulting expanded predecessors and successors, as well at the restrictions, are propagated down to the leaf tasks.

### 2.2 Final Phase

The final phase processes a task list that is a concatenation of task lists produced from the initial phase, or just the one task list already processed by the initial phase. As such, it's working with task copies for which all variables have been expanded and eliminated. The processing for this phase augments the substitution map by analyzing the task list for structural duplicates. It then applies those substitutions, i.e., overriding and merging duplicate tasks, thereby reducing the task list before further processing.

### 2.3 Reordering

The processing of the task list, particularly task overriding and merging, affects the overall order of the task list such that it's different from the original authored order gathered from the scopes. Not only that, when multiple streams are involved, final phase processing is dealing with a concatenated list in which the tasks must be properly reordered. To support that, each task has an intrinsic priority; the task list is primarily sorted according to that priority. Each task also specifies predecessors and successors; the task list is secondarily sorted to respect that induced partial order. After these two sorting steps, the tasks in the list are modified to clear both the predecessors and successors and then the predecessors are set to form a chain that induces an overall order that's exactly this final order of sorted task list; this chain excludes variables. This chain of dependencies ensures that the final phase processing, which deals with the concatenated task lists, will properly interleave the tasks (because of the priority sorting) while also respecting the per-stream order of the multiple streams.

### 2.4 Trigger Filtering

Each task that excludes the current trigger is removed from the task list. Note that the task list gathering process gathers all tasks because the task list is analyzed to determine which tasks need to be installed for all possible triggers. So for bootstrap bootstrap trigger, even the tasks that can't execute until they're running in an installed product are analyzed to ensure that, once the product is installed, the tasks that will need to perform in that installation, i.e., for startup or manual trigger, are properly installed. The processing of all tasks also implies that at bootstrap time, all variables that will be needed in the running installed product will be prompted early and hence will already be available in the running installed product.

### 2.5 Consolidation

The final task list processing step removes all variables from the task list and consolidates each remaining task. At this point, the tasks in the list are ready to be performed.

Copyright (c) 2014 Eike Stepper (Loehne, Germany) and others.
All rights reserved. This program and the accompanying materials are made available under the terms of the Eclipse Public License v2.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v20.html