4.3. Configuration Sources

4.3.1. Project Structure

MQC provides a default project structure if no Project Structure Source has been imported. The default project structure is based on the imported data:

• imported artifacts are shown with their names as read from the data reports, except a proper artifact mapping pattern was added (see Artifact Mapping Patterns)

• a default milestone period is used, which is starting from the date of the first imported data report and which goes until the date of the last imported data report

4.3.1.1. Project Name

MQC allows to define a dedicated project name.

Listing 4.1 Project name definition in YAML

Name: ProjectStructure
Project: SDOACar

In Excel, the project name configuration is done in a separate sheet called ProjectArtifactStructure.

Figure 4.12 Project name definition in Excel

4.3.1.2. Milestones

Milestones in MQC are treated as a time period. A milestone configuration consists of:

• MilestoneName

  The name of the milestone.

• MilestoneStartDate

  The start date of the milestone period. Either, the start date of the project for the first milestone, or one day after the end date of the previous milestone.

• MilestoneDueDate

  The end date of the milestone period respectively the date of the milestone itself.

• MilestoneDuration

  The milestone period duration.

Listing 4.2 Definition of milestones in YAML

Milestones:
- Name: Model Unit Start
  StartDate: 2021-04-27
  DueDate: 2021-06-28
  Duration: 63
- Name: Model Unit Extension
  StartDate: 2021-06-29
  DueDate: 2021-08-16
  Duration: 49
- Name: Model Integration
  StartDate: 2021-08-17
  DueDate: 2021-09-27
  Duration: 42
- Name: Test Improvement
  StartDate: 2021-09-28
  DueDate: 2021-10-18
  Duration: 21

In Excel, the milestone configuration is done in a separate sheet called ProjectMilestoneStructure.

Figure 4.13 Definition of milestones in Excel

It is not necessary to specify each of the parameters MilestoneStartDate, MilestoneDueDate, and MilestoneDuration.

Instead, it is sufficient to specify a project start date as start date of the first milestone phase in combination with a duration for all milestones. It is also possible to define start and end date per milestone without duration.

4.3.1.3. Milestone Structures

Milestones can be defined in sets, which can be organized in a tree structure, so you can easily navigate a lot of milestone sets and quickly enable or disable them. If a project has no need for a milestones structure, you can configure them as explained in Milestones.

Each set of milestones is a consecutive list of dates. Milestones from different sets can overlap or even have the same date.

When using YAML (or JSON) as configuration format:

The milestone structure consists of one or more milestone sets that contain:

• Name

  The name of the milestone set.

• Description

  An optional description of the milestone set.

• Structures

  Multiple milestone sets can be defined in the level below this set.

• Milestones

  Multiple milestones can be defined for this milestone set. The definition is identical to the unstructured Milestones.

There has to at least one Structure and/or Milestone defined in the milestone set.

Listing 4.3 Defining milestone sets and milestones in YAML

MilestoneStructures:
  - Name: 'Process'
    Milestones:
      - Name: Model Unit Start
        StartDate: 2021-04-26
        DueDate: 2021-06-27
      - Name: Model Unit Extension
        DueDate: 2021-08-15
      - Name: Model Integration
        DueDate: 2021-09-26
      - Name: Test Improvement
        DueDate: 2021-10-17
  - Name: 'ECUs'
    Structures:
      - Name: 'EV3C'
        Milestones:
          - Name: Model Implementation
            StartDate: 2021-08-15
            DueDate: 2021-08-25
          - Name: Test Implementation
            DueDate: 2021-09-05
          - Name: Test Refinement
            DueDate: 2021-09-14
          - Name: Test Completion
            DueDate: 2021-10-06
      - Name: 'MVS'
        Milestones:
          - Name: Model Implementation
            StartDate: 2021-07-01
            DueDate: 2021-07-08
          - Name: Test Implementation
            DueDate: 2021-07-15
          - Name: Test Refinement
            DueDate: 2021-08-05
          - Name: Test Completion
            DueDate: 2021-10-06
      - Name: 'OD'
        Milestones:
          - Name: Model Implementation
            StartDate: 2021-04-26
            DueDate: 2021-05-12
          - Name: Test Implementation
            DueDate: 2021-05-21
          - Name: Test Refinement
            DueDate: 2021-05-28
          - Name: Test Completion
            DueDate: 2021-10-06
      - Name: 'GP'
        Milestones:
          - Name: Model Implementation
            StartDate: 2021-06-01
            DueDate: 2021-06-12
          - Name: Test Implementation
            DueDate: 2021-06-20
          - Name: Test Refinement
            DueDate: 2021-06-24
          - Name: Test Completion
            DueDate: 2021-10-06

In Excel a milestone structure can be configured by adding value to the MilestonePath Column in the ProjectMilestoneStructure sheet. The MilestonePath consists of the name-hierarchy of milestone sets concatenated by a dot. (e.g. “ECUs.GP”) When using Excel as the configuration format, you cannot add a description for the milestone sets.

Figure 4.14 Defining milestone paths for milestones in excel

4.3.1.4. Artifacts

With the artifact configuration in the project structure the list of expected project artifacts is defined. A proper artifact configuration consists of:

• ArtifactName

  Mandatory, the name of the artifact.

• ArtifactWeight

  Optional, a weight to define that artifacts may have more or less influence than others. If nothing is set, a weight of 1 is used.

• ContextCategories

  Optional, or details about assigning context categories to artifacts see Assign Context Categories to Artifacts.

• ArtifactPath(s)

  Optional, a list of artifact names extracted from data sources to be treated as the same artifact (see Artifacts and Artifact Structure).

  Independent from a specific artifact mapping, MQC allows to define general artifact mapping patterns (see Artifact Mapping Patterns).

• ArtifactStructure

  For details see Artifact Structures.

Listing 4.4 Defining expected artifacts in YAML

Artifacts:
- Name: EV3Control_main
  Weight: 1
  Paths:
  - EV3Control_main
  - EV3Control_demo_ec
  - EV3Control_demo_ec/EV3Control
  Structures:
  - Path: Software.Teams.Components
    Value: SDOAC SW.Team F.Integration
- Name: ObstacleDetection
  Weight: 1
  ContextCategories:
  - MLC_CC
  Paths:
  - ObstacleDetection
  - EV3Control_demo_ec/VehicleManager/ObstacleDetection
  - ObstacleDetection_demo_ec
  - Obstacledetection_demo_ec
  - ObstacleDetection_demo_ec/ObstacleDetection
  Structures:
  - Path: Software.Teams.Components
    Value: SDOAC SW.Team F.Detection

In Excel, the list of expected artifacts is configured within the ArtifactStructure sheet.

Figure 4.15 Defining expected artifacts in Excel using

4.3.1.5. Artifact Structures

Artifact structures can be used in MQC to group artifacts, which can be assigned to a specific path of such a defined structure.

When using YAML (or JSON) as configuration format:

• Multiple hierarchies can defined.

• Each artifact can be assigned to one or multiple hierarchies.

• The hierarchies can have a different amount of levels.

• A description can be added also to structure levels.

Listing 4.5 Defining artifact structures and artifacts with structure assignments in YAML

ArtifactStructures:
- Name: Architecture
  Path: Software.Components
  Values:
    - Name: SDOAC SW
      Description: SDOA Car Software
      Values:
        - Name: Integration
          Description: Integration Model
        - Name: Control
          Description: Control Model
- Name: Responsibility
  Path: Devisions.Teams
Artifacts:
- Name: EV3Control
  Structures:
  - Path: Software.Components
    Value: SDOAC SW.Integration
  - Path: Devisions.Teams
    Value: Development.Team F
- Name: GlobalPosition
  Description: Subsystem for the position of the vehicle
  Structures:
  - Path: Software.Components
    Value: SDOAC SW.Control
  - Path: Devisions.Teams
    Value: Development.Team P

In Excel, the artifact structure configuration is done within the ArtifactStructure sheet (as part of the artifact configuration, see Artifacts).

When using Excel as configuration format:

• Only one hierarchy can be defined.

• Multiple structure levels are allowed.

• Structure levels have to be defined as columns with a column name beginning with “Structure” (e.g. ‘StructureSoftware’ for a structure level called “Software”).

• The order of the levels is defined by the order of the columns in the Excel (e.g. Software > Teams > Components).

Figure 4.16 Defining artifact structures and artifacts with structure assignments in Excel

4.3.2. Quality Model

MQC provides data source specific quality model configurations. These predefined source files can be directly chosen via the “Suggestions” tab within the Add dialog (see Import / Reload).

If no quality model was imported, MQC recognizes the used data sources (either based on the enabled adapters or from the data currently imported) and loads the corresponding quality model suggestions.

4.3.2.1. Base Measures

The base measures configured in the quality model define the set of expected data. When configuring base measures, MQC expects the following information:

• DataSourceName

  The tool/source providing a data report containing data to be imported by MQC. Examples are MXAM and TPT.

• MeasurementName

  A default measurement name, which is used if the measurement can’t be extracted from a report.

  Typically, a measurement means the set of operations done to determine the value for a measure. For example this can be the name of a specific guideline document, which is used for the static analysis of a model, or the kind of test environment like MiL, SiL or PiL.

• BaseMeasureName

  The name for a group of measures, e.g. “FindingCount” for the data source MXAM.

• VariableName

  The name of a specific measure inside a measure group, e.g. “Passed” or “Failed”. Full measure names would then be “FindingCount.Passed” or “FindingCount.Failed”.

• DefaultValue

  The value, which should be set if no value for a specific measure can be extracted from a report, typically 0. (see Default Values)

• Aggregation Combining multiple artifact paths into an artifact can result in more than one data point for the same report at a specific revision and measure. You can specify a aggregation method, sum or avg, to aggregate instead of marking one data point as duplicate and ignoring it.

4.3.2.1.1. Default Values

MQC uses the configured defaults, if the report contains values for at least one other measure belonging to the same base measure group. Otherwise, the whole base measure group is treated as “Missing”.

Listing 4.6 Defining expected base measures with there default values in YAML

BaseMeasures:
- Name: MXAM.GuidelineAnalysis.FindingCount.Aborted
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Canceled
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Failed
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Info
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Passed
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Repaired
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Review
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Unrepaired
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Warning
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Ignored
  DefaultValue: 0

In Excel, the base measure configuration is done in a separate sheet called BaseMeasure.

Figure 4.17 Defining expected base measures with there default values in Excel

4.3.2.2. Derived Measures

A fully qualified derived measure consists of:

• DataSourceName

  Optional, typically the name of the data source of the base measures used to calculate the derived measure.

• MeasurementName

  Optional, typically the name of the measurement of the base measures used to calculate the derived measure.

• DerivedMeasureName

  Mandatory, the name for a group of derived measures.

• VariableName

  Mandatory, the name of a specific derived measure inside a derived measure group.

• MeasureFunction

  Mandatory, the expression to calculate a derived measure using base measures and/or other derived measures.

  Base and derived measures must be set in square brackets [ and ]!

If no data source and measurement names are configured (short notation), MQC takes data source and measurement from the base and/or derived measures used to calculate the current derived measure.

This means, if the derived measure configuration is in short notation and the base measures used to calculate the derived measure are provided for multiple measurements (e.g. TestCount.Total for MiL and for SiL), the derived measure will be calculated for all the measurements as well.

To calculate a derived measure for a specific measurement only, fully qualified measure names have to be used inside the measure function.

Listing 4.7 Defining derived measures using a short notation in YAML

DerivedMeasures:
- Name: GuidelinesCalc.Passed
  Expression: '[GuidelineCount.Passed]+log(1+[FindingCount.Passed],2)'
- Name: GuidelinesCalc.Repaired
  Expression: '[GuidelineCount.Repaired]+log(1+[FindingCount.Repaired],2)'
- Name: GuidelinesCalc.Unrepaired
  Expression: '[GuidelineCount.Unrepaired]+log(1+[FindingCount.Unrepaired],2)'
- Name: GuidelinesCalc.Failed
  Expression: '[GuidelineCount.Failed]+log(1+[FindingCount.Failed],2)'
- Name: GuidelinesCalc.Info
  Expression: '[GuidelineCount.Passed with Infos]+log(1+[FindingCount.Info],2)'
- Name: GuidelinesCalc.Review
  Expression: '[GuidelineCount.Review]+log(1+[FindingCount.Review],2)'
- Name: GuidelinesCalc.Warning
  Expression: '[GuidelineCount.Warning]+log(1+[FindingCount.Warning],2)'
- Name: GuidelinesCalc.AbortedCanceled
  Expression: '[GuidelineCount.Aborted]+[GuidelineCount.Canceled]'

In Excel, the derived measure configuration is done in a separate sheet called DerivedMeasure.

Figure 4.18 Defining derived measures using a short notation in Excel

4.3.2.3. Quality Properties

When configuring quality properties, MQC expects the following information:

• QualityPropertyName

  Mandatory, the name of the quality property.

• Weight

  Optional, a weight to define that quality properties may have more or less influence than others. If nothing is set, a weight of 1 is used.

• MeasurementFunction

  Mandatory, the expression to calculate a quality property measure value using base measures and/or other derived measures.

  Base and derived measures must be set in square brackets [ and ]!

• Description

  Optional, an explanation of the quality property may be added to document and visualize the background concept.

If the measurement function is defined in short notation and the base and derived measures used to calculate the quality property are provided for multiple measurements (e.g. TestCount.Total for MiL and for SiL), the quality property will be calculated for all the measurements as well.

To calculate a quality property for a specific measurement only, fully qualified measure names have to be used inside the measurement function.

Listing 4.8 Quality property definition in YAML

QualityProperties:
- Name: Testable Requirements with Test Sequences
  Weight: 2
  Expression: [Testable Requirements with Test Sequences.Reached] /
              [Testable Requirements with Test Sequences.Total]
  Description: >
     Show the percentage of the reached testable requirements
     with test Sequences

In Excel, the quality property configuration is done within the QualityModel sheet (together with the quality structure configuration, see Quality Structures).

Figure 4.19 Quality property definition in Excel

4.3.2.4. Bin Conditions

When configuring bin conditions, attention must be paid to the following:

• MeasurementFunction

  Instead of a numerical function using base measures and/or derived measures, the term [MQC.Bin] must be used to indicate that the quality property is calculated via a bin condition.

• BinConditions

  For each quality bin a conditional expression must be added. The particular quality bin is assigned, if the expression returns true.

  Bin conditions must be defined in the right order starting with the lowest quality bin up to the highest. This is important for the evaluation of the conditions.

  Please, ensure that all used quality bins are properly defined (see Quality Bins).

Listing 4.9 Using Bin Conditions to define quality properties in YAML

QualityProperties:
- Name: B2B
  Weight: 1
  Expression: '[MQC.Bin]'
  BinConditions:
  - Name: Bad
    Expression: >
      [B2B Statement Coverage.Ratio] is not null and
      [B2B MC/DC Coverage.Ratio] is not null and
      [B2B Test Count.Failed] is not null and
      [B2B Test Count.Error] is not null
  - Name: Acceptable
    Expression: False
  - Name: Good
    Expression: >
      [B2B Statement Coverage.Ratio] = 1 and
      [B2B Test Count.Failed] = 0 and
      [B2B Test Count.Error] = 0

In Excel, the bin condition configuration is split into two parts. The quality property configuration still has to be done within the QualityModel sheet.

Figure 4.20 Quality property definition using bin conditions instead of a measurement function in Excel

Besides, the bin condition configuration is done in a separate sheet called Bin Condition (see Figure 4.20).

4.3.2.5. Quality Structures

To group quality properties in order to define different quality aspects, quality structures can be defined.

Quality properties can be assigned to a specific path of such a defined structure.

When using YAML (or JSON) as configuration format:

• Multiple hierarchies can defined.

• Each quality property can be assigned to one or multiple hierarchies.

• The hierarchies can have a different amount of levels.

• A description can be added also to structure levels, not only to quality properties.

Listing 4.10 Definition of a quality model structure and assigning quality properties to this structure in YAML

QualityStructures:
  - Name: Functional Suitability
    Path: Models.Characteristics.Subcharacteristics
    Values:
      - Name: Product Quality Model
        Values:
          - Name: Correctness
            Description: >
               Grouping the quality properties.

               Quality properties in this group evaluate the successful
               operation of the models.
            Values:
              - Name: Model Design
              - Name: Functional Requirements
                Description: >
                   Grouping the quality properties that evaluate the services,
                   the models must offer
              - Name: Test Sequences
              - Name: Assessments
          - Name: Completeness
            Values:
              - Name: Functional Requirements
              - Name: Test Sequences
              - Name: Assessments
              - Name: Model Coverage
              - Name: Code Coverage
  - Name: Quality Assurance Methods
    Path: Quality Assurance.QA Method.QA Work Product
    Values:
      - Name: Product Quality Model
        Values:
          - Name: Static Analysis
            Values:
              - Name: Guidelines Model Design
              - Name: Guidelines Model Architecture
          - Name: Test
            Values:
              - Name: Requirements
              - Name: Test Sequences
              - Name: Assessments
              - Name: Structural Coverage
QualityProperties:
  - Name: Requirements Compliance
    Weight: 3
    Expression: '[Requirements Compliance.Reached] / [Requirements Compliance.Total]'
    Structures:
      - Path: Models.Characteristics.Subcharacteristics
        Value: Product Quality Model.Correctness.Functional Requirements
      - Path: Quality Assurance.QA Method.QA Work Product
        Value: Product Quality Model.Test.Requirements

In Excel, the quality structure configuration is done within the QualityModel sheet.

When using Excel as configuration format:

• Only one hierarchy can be defined.

• Multiple structure levels are allowed.

• Structure levels have to be defined as columns with a column name beginning with “Quality” (e.g. ‘QualityCharacteristics’ for a structure level called “Characteristics”).

• The order of the levels is defined by the order of the columns in the Excel (e.g. Models > Characteristics > Subcharacteristics).

Figure 4.21 Definition of a quality model structure and assigning quality properties to this structure in Excel

4.3.2.6. Context Categories

For details about defining context categories see Configuration of Context Categories.

4.3.2.7. Actions

For details about actions, see Actions.

Listing 4.11 Action definition in YAML

QualityProperties:
- Name: Testable Requirements with Test Sequences
  Weight: 2
  Expression: [Testable Requirements with Test Sequences.Reached] /
              [Testable Requirements with Test Sequences.Total]
  Actions:
  - Link test sequences to (covered) requirements
  - Derive further test sequences from uncovered requirements

In Excel, the action configuration is done in a separate sheet called ActionList.

Figure 4.22 Defining recommended actions for improving quality properties in Excel

4.3.3. Context Categories

Details about how to switch on or off categories in MQC can be found in Context Categories.

4.3.3.1. Configuration of Context Categories

Context categories have to be configured in the quality model (see Quality Model).

A context category definition may contain:

• name of the context category

  mandatory, later used to be assigned to artifacts

• expected data

  optional, white list

• excluded data

  optional, black list

Both, expected and excluded data has to be defined using comma-separated lists of data sources, measurements and/or measures. Use either the full name or name patterns (including wildcards).

Listing 4.12 Configuring context categories in YAML

ContextCategories:
- Name: StaticAnalysisOnly
  Expect: MXAM, MXRAY
- Name: TestNoCoverage
  Expect: MXAM, MXRAY, MTest
  Exclude: MTest.*.Model*Coverage

In Excel, the context category configuration is done in a separate sheet called ContextCategory.

Figure 4.23 Configuring context categories in Excel

An empty “Expect” field means, all data is expected for that context category. To state that no data is expected at all, use the key word “none”.

An empty “Exclude” field means, nothing is excluded, but still, the amount of data may be reduced by specifying expected data.

4.3.3.2. Assign Context Categories to Artifacts

Context categories can be assigned to one or multiple artifacts in the project structure configuration source (see Artifacts).

Listing 4.13 Assign context categories to artifacts in YAML

 Artifacts:
   - Name: EV3Control_main
   - Name: ObstacleDetection
     ContextCategories:
       - MLC_CC
   - Name: GlobalPosition
     ContextCategories:
       - Global_CC
   - Name: ManageVehicleStates
     ContextCategories:
       - MLC_CC

In Excel, the assignment of context categories to artifacts is done within the ArtifactStructure sheet.

Figure 4.24 Assign context categories to artifacts in Excel

When no context category is assigned, i.e. the field is left empty, all available data is shown for that artifact.

4.3.4. Target Values

To configure target values in MQC, the following information is necessary:

• ArtifactName

  The name of the artifact a target value shall be applied to.

• MilestoneName

  The name of the milestone a target value shall be reached at.

• MeasureName[.TargetName]

  The name of the measure, data or quality property, a target value shall be applied to, which is optionally followed by a specific target value name:

  • Targets for data measures (fully qualified):

    DataSourceName.MeasurementName.BaseMeasureName.VariableName[.TargetName]

  • Targets for quality properties:

    QualityPropertyName[.TargetName]

• TargetValue

  The value, e.g. a percentage value for quality properties, that is expected to be reached at the date of the configured milestone.

If the configured measure name does not contain a target name at the end, a default target name “Target” is applied by MQC instead.

It is not necessary to define a target for each configured milestone (see Calculation of Target Values per Revision).

Listing 4.14 Defining target values per measure, per artifact and per milestone in YAML

Artifacts:
- Name: ManageVehicleStates
  Milestones:
  - Name: Model Unit Start
    Targets:
    - Name: Guideline Compliance (GuidelineAnalysis)
      Value: 80
    - Name: Requirements Compliance
      Value: 60
  - Name: Model Unit Extension
    Targets:
    - Name: Guideline Compliance (GuidelineAnalysis)
      Value: 80
    - Name: Requirements Compliance
      Value: 80
- Name: EV3Control_main
  Milestones:
  - Name: Model Unit Start
    Targets:
    - Name: Guideline Compliance (GuidelineAnalysis)
      Value: 0
      - Name: Requirements Compliance
        Value: 0
    - Name: Requirements with Reviewed Testability
      Value: 0

In Excel, the target value configuration is done in a sheet called Target Values.

Figure 4.25 Defining target values per measure, per artifact and per milestone in Excel

Details about how to switch on or off targets in MQC can be found in Target values in visualizations (Custom pages).

4.3.5. Annotations

For a proper description of necessary configuration parameters, refer to Annotations.

Mandatory parameters are:

• Title

• Artifact

• Quality Property

Listing 4.15 Defining annotation source in YAML

Authors:
- Model Engineering Solutions GmbH
Annotations:
- Title: 80% because halted for technical reasons
  Author: mqc_admin
  Artifact: EV3Control_main
  QualityProperty: Code Condition Coverage
  ValidFrom: 2021-10-11
  ValidTo: 2021-10-17
  ConditionType: Bin
  ConditionBins:
    - Acceptable
  TargetType: Quality
  TargetValue: 80.00
- Title: Bad because not enough tests
  Author: mqc_admin
  Artifact: GlobalPosition
  QualityProperty: Testable Requirements with Test Sequences
  ValidFrom: 2021-10-11
  ValidTo: 2021-10-17
  ConditionType: Bin
  ConditionBins:
    - Good
  TargetType: Bin
  TargetValue: Bad
- Title: Because Model Coverage - Decision under 80%
  Description: see Data Origins
  Author: mqc_admin
  Artifact: ObstacleDetection
  QualityProperty: Model Decision Coverage
  ValidFrom: 2021-10-11
  ValidTo: 2021-10-17
  ConditionType: Bin
  ConditionBins:
    - Acceptable
  TargetType: Quality
  TargetValue: 75.00

All other configurations are optional and can be added respectively edited later directly in MQC (see Annotations).