14. Quality Model Configuration

As described in Quality Concept, MQC handles the imported data in such a way that quality can be structured and computed out of it. You can configure the quality model by importing one or multiple Quality Model source files, which can have the YAML or Excel format. These files include the definition of measurement functions to calculate measures for quality properties as well as the definition of base measures and derived measures.

14.1. Importing Quality Model source files

To import a Quality Model source file into your project, press the Add button next to Quality Model Sources in the Quality Model dialog (see Figure 14.1).


MQC allows to import multiple Quality Model source files. By that it is possible to define measures and quality properties for each data source in its own Quality Model source file.

When adding a new Quality Model source file, the content is validated for consistency and, additionally, it is checked for conflicts against the already existing Quality Model configuration defined by the previously added files.

All changes done via the Quality Model dialog will be applied to the project first after pressing the Update Project button.


If you do not have a proper Quality Model source file that can be adapted, you can export one of the currently imported Quality Model source files (see Exporting Quality Model source files) and use it as a template in which you define your own project’s quality model.

It is possible to reload or remove each Quality Model source file individually as well as all source files at once.

MQC already provides data source specific Quality Model configurations. These predefined Quality Model configurations can be directly chosen via the “Suggestions” tab within the Add dialog (see Figure 14.2).

Additionally, MQC uses these Quality Model source files if no Quality Model at all was added by the user. In this case MQC recognizes the data sources used from the data currently imported and loads the corresponding provided initial quality models.


Figure 14.2 Predefined Quality Models per data source shipped with MQC

14.2. Exporting Quality Model source files

In the Quality Model dialog MQC allows to export each Quality Model source file individually by pressing the Export button next to a Quality Model source in the list of source files (see Figure 14.1). This will directly export a Quality Model source file of the file type is was originally imported in.

If you would like to merge the Quality Model configurations from multiple imported source files into a single Quality Model source file, press the Export multiple button at the top. This opens an additional Export dialog as shown in Figure 14.3. With this it is possible to select the relevant source files to be combined.

Additionally, this dialog provides the option to explicitly choose the file type of the export.


Figure 14.3 Export dialog for exporting multiple Quality Model sources

14.3. Quality Model Configuration

14.3.1. Base Measures and Default Values

Base Measures are the imported values provided by your data sources. As described later in Derived Measures and Quality Properties, base measures are used to calculate other measures, so-called derived measures, and – most important – quality measures, which are the base to compute quality.

For configuring base measures, MQC expects the following structure:

  • data source name

  • (default) measurement name

  • base measure name

  • variable name

  • default value

The data source is the tool, which is used to provide data to be imported into MQC. Examples are MXAM and MTest.

Typically, a measurement means the set of operations done to determine the value for a measure. For example this can be the name of a specific guideline document, which is used for the static analysis of a model, or the kind of test environment like MiL, SiL or PiL.

If a tool adapter supports the possibility to provide measurement names out of data files, the default is ignored and MQC uses the name provided by the tool adapter/data file.

Without default values, in case there are values missing for variables inside such a base measure group for a certain revision, a measurement function using these missing base measure variables could not be calculated. Therefore, MQC uses a configured default in case the data source has provided values for some base measure variables of a base measure group but not for all.

As an example, please imagine the base measure group FindingCount, which contains the variables Canceled, Aborted, Review, Failed, Unrepaired, Warning, Info, Repaired, Passed and Ignored. If for a certain revision the data source does not provide values for e.g. Repaired and Passed, MQC uses the default values for the related base measure variables as defined in the Quality Model source. Hence, a corresponding measurement function can be calculated.

Listing 14.1 Defining expected base measures with there default values in YAML
- Name: MXAM.GuidelineAnalysis.FindingCount.Aborted
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Canceled
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Failed
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Info
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Passed
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Repaired
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Review
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Unrepaired
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Warning
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Ignored
  DefaultValue: 0

Figure 14.4 Defining expected base measures with there default values in Excel


Please note, that all the expected base measure variables have to be defined in the Quality Model sources. This is to assure that missing values can be substituted with the default values.

In addition, MQC uses default values only if a base measure variable inside a base measure group is missing! If the complete base measure is missing for a revision, no default values will be used!

In the case when base measures are completely missing in the current revision but values exist for any of the previous revisions, data propagation may be used to gain a full set of available information (see Data Propagation).

14.3.2. Derived Measures

Derived measures are computed either from base measures or from other derived measures as explained in Derived Measures.

This may be used to shorten the definition of measurement functions, which compute quality measures for quality properties (see Quality Properties).

A fully qualified derived measure name consists of four parts (same structure as for base measures):

  • data source name

  • measurement name

  • derived measure name

  • variable name

With that you are able to define groups of derived measures, each group containing multiple variables.

The configuration of data source name and measurement name is optional. If nothing is set here (short notation), MQC takes data source and measurement from the base and/or derived measures used to calculate the derived measure.

Listing 14.2 Defining derived measures using a short notation (derived measure name and variable name only) in YAML
- Name: GuidelinesCalc.Passed
  Expression: '[GuidelineCount.Passed]+log(1+[FindingCount.Passed],2)'
- Name: GuidelinesCalc.Repaired
  Expression: '[GuidelineCount.Repaired]+log(1+[FindingCount.Repaired],2)'
- Name: GuidelinesCalc.Unrepaired
  Expression: '[GuidelineCount.Unrepaired]+log(1+[FindingCount.Unrepaired],2)'
- Name: GuidelinesCalc.Failed
  Expression: '[GuidelineCount.Failed]+log(1+[FindingCount.Failed],2)'
- Name: GuidelinesCalc.Info
  Expression: '[GuidelineCount.Passed with Infos]+log(1+[FindingCount.Info],2)'
- Name: GuidelinesCalc.Review
  Expression: '[GuidelineCount.Review]+log(1+[FindingCount.Review],2)'
- Name: GuidelinesCalc.Warning
  Expression: '[GuidelineCount.Warning]+log(1+[FindingCount.Warning],2)'
- Name: GuidelinesCalc.AbortedCanceled
  Expression: '[GuidelineCount.Aborted]+[GuidelineCount.Canceled]'

Figure 14.5 Defining derived measures using a short notation (derived measure name and variable name only) in Excel

Listing 14.2 shows the definition of several derived measures that are computed from various variables of the base measures FindingCount and GuidelineCount provided by the data source MXAM. For example, the derived measure GuidelinesCalc.Passed is calculated by the following measure function:

[GuidelineCount.Passed] + log(2, 1 + [FindingCount.Passed])


As shown in the above example, base and derived measures used in a measure function must be set in square brackets [ and ]!

If measure functions are defined using a short notation for measure names only consisting of base / derived measure name and variable name (see Listing 14.2), MQC checks for matching measures that were imported and replaces the used measure names by the fully qualified names.

If a measure function contains a base measure that exists for different measurements, for example if FindingCount.Passed was measured multiple times by checking compliance according to different guideline documents, MQC calculates the corresponding derived measure for each of these measurements. The same applies to measure functions containing derived measures calculated for multiple measurements.


If a derived measure is calculated for multiple measurements, the configured data source and measurement names will be replaced by data source and measurement from the base and/or derived measures used to calculate the derived measure.

To calculate a derived measure for a selected measurement only, please, use fully qualified measure names when defining the measure function.

14.3.3. Quality Properties

Quality properties are the lowest level of computable quality. For each quality property a measurement function has to be defined that is used to calculate a quality measure between 0 and 1 by using base and derived measures.

Derived measures (see Derived Measures) can be used instead of base measures to simplify quality computation, hence, to shorten the formula of a particular measurement function.

Listing 14.3 shows an example for a measurement function, where all base measure variables for the base measures FindingCount and GuidelineCount were replaced by the defined derived measures.

Optionally, for a quality property you can define actions, which are recommended tasks for improving the corresponding quality property value. For detailed information about Actions in MQC please refer to chapter Actions.

Listing 14.3 Quality Property definition in YAML
- Name: Guideline Compliance
  Weight: 1
  Expression: (1.0*([GuidelinesCalc.Passed]+[GuidelinesCalc.Info]+[GuidelinesCalc.Repaired])
              + 0.8*[GuidelinesCalc.Warning] + 0.5*[GuidelinesCalc.Unrepaired]
              + 0.2*[GuidelinesCalc.Failed] + 0.1*[GuidelinesCalc.Review]
              + 0.0*[GuidelinesCalc.AbortedCanceled])
              / ([GuidelinesCalc.Passed] + [GuidelinesCalc.Info] + [GuidelinesCalc.Repaired]
              + [GuidelinesCalc.Warning] + [GuidelinesCalc.Unrepaired] + [GuidelinesCalc.Failed]
              + [GuidelinesCalc.Review] + [GuidelinesCalc.AbortedCanceled])
  - Refactor model according to guideline findings

Figure 14.6 Quality Property definition in Excel (without Action definition)


In Excel you will have to define the actions separately in the Action-List sheet (see Figure 14.7).


Figure 14.7 Defining recommended actions for improving quality properties in Excel

As for measure functions to calculate derived measures, you can define quality property measurement functions using a short notation for measure names consisting of base / derived measure name and variable name. MQC then checks for matching measures that were imported (base measures) or calculated (derived measures) and replaces the used measure names by the fully qualified names.


As shown in the above example, base and derived measures used in a measurement function must be set in square brackets [ and ]!

If a measurement function uses a base or derived measure that exists for different measurements, for example if GuidelinesCalc.Passed was calculated multiple times, MQC calculates the corresponding quality property for each of the measurements.


If a quality property is calculated for multiple measurements, the configured quality property name will be extended by the name of the measurement, e.g. Guideline Compliance (GuidelineAnalysis), whereas Guideline Compliance is the defined quality property name and GuidelineAnalysis is the name of the measurement.

To calculate a quality property for a selected measurement only, please use fully qualified measure names when defining the measurement function.

14.3.4. Quality Model Structure

As explained in Quality Model Structure, quality in MQC is aggregated on different levels (as shown in Figure 6.4). MQC allows to assign quality properties with their measurement functions to sub-characteristics. These sub-characteristics again can be assigned to characteristics (see Figure 14.8). This enables the user to group quality properties in order to define different quality aspects, which may be analyzed separately.

Listing 14.4 Definition of a quality model structure and assigning quality properties to this structure in YAML
- Name: Product Quality Model
  - Name: Correctness
    - Name: Model Design
      - Guideline Compliance
    - Name: Functional Requirements
      - Requirements Compliance
    - Name: Test Sequences
      - Test Sequence Compliance
    - Name: Assessments
      - Assessments Compliance
  - Name: Completeness
      - Name: Functional Requirements
        - Requirements with Reviewed Testability
        - Testable Requirements with Assessments
        - Testable Requirements with Test Sequences
      - Name: Test Sequences
        - Test Sequence Work Progress
        - Reviewed Test Sequences
      - Name: Assessments
        - Assessment Work Progress
        - Reviewed Assessments
      - Name: Model Coverage
        - Model Condition Coverage
        - Model Decision Coverage
      - Name: Code Coverage
        - Code Condition Coverage
        - Code Decision Coverage
        - Code Statement Coverage

Figure 14.8 Definition of a quality model structure and assigning quality properties to this structure in Excel

The following sunburst visualization (Figure 14.9, lower right) illustrates the computation of the configured quality model.


Figure 14.9 Computation of the overall quality illustrated using the sunburst visualization.

The outer ring of the sunburst consists of all quality properties defined by the quality model. Please note, each quality property is already aggregated over the artifacts (see Figure 6.6 in section Quality Aggregation).

The next ring shows the sub-characteristics defined by the quality model. A sub-characteristic is calculated using the average of all quality properties assigned to that sub-characteristic.

The third ring of the sunburst chart shows the configured characteristics. Each characteristic is calculated using the average of all quality properties assigned to all sub-characteristics belonging to that particular characteristic.

Finally, the inner circle shows the overall quality.


By default, MQC uses all quality properties to compute the separate levels of the quality model, even if the measure value for a certain quality property cannot be calculated. This may be the case if all or a few measures, which are used to compute a quality property measurement function, are not available within a certain revision, i.e. not yet measured.

Missing quality properties are always treated as 0 when calculating the overall quality of a project.

MQC allows to exclude missing quality properties for the calculation. To achieve this, open the Settings dialog in the Configuration panel and enable the Exclude missing values check box under Quality calculation title. This may be used to compute quality only for these parts of a project, which are already available, respectively to compute quality only based on these measures already provided.

In the example shown in Figure 14.9 all quality properties defined by the quality model are treated equally. This means they have the same weight when computing the overall quality of a project. Nevertheless, MQC allows to give some quality properties a higher importance than others. This relative importance can be defined in the Quality Property weight.

Each level of the quality model will use the weighted average for quality computation.

14.3.5. Context Categories

By using context categories you can define, which data (on the level of base measures) is expected for a certain artifact. In that way, data never provided and therefore normally stated as “missing” - and probably resulting in bad quality - is treated as “excluded” data. This means it is excluded in availability and quality calculation.

Configuration of context categories is done in two steps:

  • assigning data to (or explicitly removing from) a context category is done within the Quality Model source, because this is directly related to the base measures defined here

  • assigning context categories to artifacts is done in the project structure configuration file, where the expected artifacts are defined (see Assign Context Categories to Artifacts)

For each context category you can define, which data is expected (white list) respectively which data should be excluded (black list). A combination of both is possible (see Figure 14.10).

Listing 14.5 Configuring context categories in YAML
- Name: StaticAnalysisOnly
  Expect: MXAM, MXRAY
- Name: TestNoCoverage
  Expect: MXAM, MXRAY, MTest
  Exclude: MTest.*.Model*Coverage

Figure 14.10 Configuring context categories in Excel

Expected as well as excluded data has to be defined using comma-separated lists of data sources, measurements and/or measures. Use either the full name or name patterns (including wildcards).

In the given example two context categories are defined. The first one only includes all data from MXAM and MXRAY. For the second context category MTest is configured as additional data source, but without the two measures Model Condition Coverage and Model Decision Coverage.


If you leave the “Expect” field empty, all data is expected for that context category. To state that no data is expected at all, use the key word “none”.

If the “Exclude” field is left empty, this means nothing is excluded, but still, the amount of data may be reduced by specifying expected data.

For a detailed explanation how to use context categories, please refer to chapter Context Categories.

14.4. Quality Model for MES data sources

14.4.1. Quality Model for MXAM

In section Quality Properties there were already examples shown, how to define derived measures and quality properties, as well as a measurement function using measures provided by MXAM.

The defined derived measures are calculated using both, guideline results (GuidelineCount) as well as check findings (FindingCount). Because of the fact that the amount of check results typically is much higher than the number of the guideline results, we use a log function to reduce the impact of the check findings.

GuidelinesCalc.Passed = [GuidelineCount.Passed] + log(1 + [FindingCount.Passed], 2)

The MXAM quality model contains derived measures for all possible results, i.e. Failed, Info, Warning and so forth.

These derived measures are used in the measurement function for the quality property Guideline Compliance, which is defined as follows:

(1.0 * ([GuidelinesCalc.Passed] + [GuidelinesCalc.Info] + [GuidelinesCalc.Repaired]) +
0.8 * [GuidelinesCalc.Warning] +
0.5 * [GuidelinesCalc.Unrepaired] +
0.2 * [GuidelinesCalc.Failed] +
0.1 * [GuidelinesCalc.Review] +
0.0 * [GuidelinesCalc.AbortedCanceled]) /
([GuidelinesCalc.Passed] + [GuidelinesCalc.Info] + [GuidelinesCalc.Repaired] +
[GuidelinesCalc.Warning] + [GuidelinesCalc.Unrepaired] + [GuidelinesCalc.Failed] +
[GuidelinesCalc.Review] + [GuidelinesCalc.AbortedCanceled])

As you can see, the measurement function calculates the weighted average of all derived measures.

There are two additional MXAM derived measures defined.

IssueCount.Value which is the sum of all “bad” findings, i.e.

  • FindingCount.Canceled

  • FindingCount.Aborted

  • FindingCount.Review

  • FindingCount.Failed

  • FindingCount.Unrepaired

  • FindingCount.Warning

SevereIssueCount.Value which is the sum of all “bad” findings not considering FindingCount.Warning.

14.4.2. Quality Model for M-XRAY

The MXRAY quality model contains quality properties based on base measures provided by the data source M-XRAY.

As an example, the measurement function for Local Complexity is defined as follows:

(1.0 * [Local Complexity.Good] +
0.8 * [Local Complexity.Acceptable] +
0.2 * [Local Complexity.Bad]) /
([Local Complexity.Good] + [Local Complexity.Acceptable] + [Local Complexity.Bad])

The same pattern may be used for Modelling Depth, Level, Inports, Outports as well as for %Elementary Inputs Unused (globally).

14.4.3. Quality Model for MTest

The MTest quality properties are defined by comparing current and expected results, which leads to quite simple measurement functions, e.g. the quality property Requirements with Reviewed Testability is calculated as follows:

[Requirements with Reviewed Testability.Reached] / [Requirements with Reviewed Testability.Total]

Following this pattern, the MTest quality model additionally defines quality properties for:

  • Requirements Compliance

  • Testable Requirements with Assessments

  • Testable Requirements with Test Sequences

  • Test Sequence Work Progress

  • Reviewed Test Sequences

  • Test Sequence Compliance

  • Assessment Work Progress

  • Assessments Compliance and

  • Reviewed Assessments.

14.5. Measurement function definition

This section aims to give you an overview on how to define your own quality properties or adapt your measurement functions.

You can use a bunch of mathematical functions provided within MQC. Apart from the four basic arithmetic operations (+, -, * and /), MQC allows to use: Abs, Exp, Ceiling, Floor, Round, Ln, Log, Mod, Power, Product, Sqrt, Min, Max, Avg, WeightedAverage, Case... when and IF... else.

In the following some of these functions are explained with their possible usage within a measurement function.

We already have seen

  • Log (2, n):

    Given the case you only have at your disposal certain base measure values, for that your quality property calculation tends to converge to 1 using basic arithmetic operations. Here you might consider to use Log function with basis 2 (please see the derived measure calculation in the example above Quality Model for MXAM.)


For MQC 5.0 and lower, the log function is defined as log (n, 2).

Given the case that you want to avoid negative values in your measurement function, you can handle this at any time of its calculation using the

  • Abs-function, that returns the absolute value of the argument.

Abs([GuidelinesPassedCalc] - [GuidelinesInfoCalc])


  • Case... when:

  when ([GuidelinesPassedCalc] - [GuidelinesInfoCalc] < 0) then 1
  when ([GuidelinesPassedCalc] - [GuidelinesInfoCalc] > 0) then ([GuidelinesPassedCalc] - [GuidelinesInfoCalc])
  else 0
  • Power

    returns Arg1 raised to the Arg2 power.

Power(Arg1, Arg2)
  • Exp

    Another useful application of mathematical functions could be the case of using the Exp (argument) function, which returns e raised to the power of the argument.

    A linear function, such as in the first example (see below) might not be feasible to accurately adapt the Quality Property of ModelClones to your needs. Therefore, the second example shows a better function:

1.0 - ([ModelClones.Clones] / [ModelClones.Subsystems])
1.0 / Exp( 2 * [ModelClones.Clones] / [ModelClones.Subsystems])
  • Avg

    This function calculates the average (mean) value of the input parameters:

Avg(parameter 1, parameter 2, ..., parameter n)