4.3. Configuration Sources

4.3.1. Project Structure

MQC provides a default project structure if no Project Structure Source has been imported. The default project structure is based on the imported data:

  • imported artifacts are shown with their names as read from the data reports, except a proper artifact mapping pattern was added (see Artifact Mapping Patterns)

  • a default milestone period is used, which is starting from the date of the first imported data report and which goes until the date of the last imported data report

4.3.1.1. Project Name

MQC allows to define a dedicated project name.

Listing 4.1 Project name definition in YAML
Name: ProjectStructure
Project: SDOACar

In Excel, the project name configuration is done in a separate sheet called ProjectArtifactStructure.

../../_images/MQC_ConfigProjectStructure_ProjectNaming.png

Figure 4.12 Project name definition in Excel

4.3.1.2. Milestones

Milestones in MQC are treated as a time period. A milestone configuration consists of:

  • MilestoneName

    The name of the milestone.

  • MilestoneStartDate

    The start date of the milestone period. Either, the start date of the project for the first milestone, or one day after the end date of the previous milestone.

  • MilestoneDueDate

    The end date of the milestone period respectively the date of the milestone itself.

  • MilestoneDuration

    The milestone period duration.

Listing 4.2 Definition of milestones in YAML
Milestones:
- Name: Model Unit Start
  StartDate: 2021-04-27
  DueDate: 2021-06-28
  Duration: 63
- Name: Model Unit Extension
  StartDate: 2021-06-29
  DueDate: 2021-08-16
  Duration: 49
- Name: Model Integration
  StartDate: 2021-08-17
  DueDate: 2021-09-27
  Duration: 42
- Name: Test Improvement
  StartDate: 2021-09-28
  DueDate: 2021-10-18
  Duration: 21

In Excel, the milestone configuration is done in a separate sheet called ProjectMilestoneStructure.

../../_images/MQC_ConfigProjectStructure_ProjectMilestoneStructure.png

Figure 4.13 Definition of milestones in Excel

It is not necessary to specify each of the parameters MilestoneStartDate, MilestoneDueDate, and MilestoneDuration.

Instead, it is sufficient to specify a project start date as start date of the first milestone phase in combination with a duration for all milestones. It is also possible to define start and end date per milestone without duration.

4.3.1.3. Artifacts

With the artifact configuration in the project structure the list of expected project artifacts is defined. A proper artifact configuration consists of:

  • ArtifactName

    Mandatory, the name of the artifact.

  • ArtifactWeight

    Optional, a weight to define that artifacts may have more or less influence than others. If nothing is set, a weight of 1 is used.

  • ContextCategories

    Optional, or details about assigning context categories to artifacts see Assign Context Categories to Artifacts.

  • ArtifactPath(s)

    Optional, a list of artifact names extracted from data sources to be treated as the same artifact (see Artifacts and Artifact Structure).

    Independent from a specific artifact mapping, MQC allows to define general artifact mapping patterns (see Artifact Mapping Patterns).

  • ArtifactStructure

    For details see Artifact Structures.

Listing 4.3 Defining expected artifacts in YAML
Artifacts:
- Name: EV3Control_main
  Weight: 1
  Paths:
  - EV3Control_main
  - EV3Control_demo_ec
  - EV3Control_demo_ec/EV3Control
  Structures:
  - Path: Software.Teams.Components
    Value: SDOAC SW.Team F.Integration
- Name: ObstacleDetection
  Weight: 1
  ContextCategories:
  - MLC_CC
  Paths:
  - ObstacleDetection
  - EV3Control_demo_ec/VehicleManager/ObstacleDetection
  - ObstacleDetection_demo_ec
  - Obstacledetection_demo_ec
  - ObstacleDetection_demo_ec/ObstacleDetection
  Structures:
  - Path: Software.Teams.Components
    Value: SDOAC SW.Team F.Detection

In Excel, the list of expected artifacts is configured within the ArtifactStructure sheet.

../../_images/MQC_ContextCategories_ConfigProjectStructure.png

Figure 4.14 Defining expected artifacts in Excel using

4.3.1.4. Artifact Structures

Artifact structures can be used in MQC to group artifacts, which can be assigned to a specific path of such a defined structure.

When using YAML (or JSON) as configuration format:

  • Multiple hierarchies can defined.

  • Each artifact can be assigned to one or multiple hierarchies.

  • The hierarchies can have a different amount of levels.

  • A description can be added also to structure levels.

Listing 4.4 Defining artifact structures and artifacts with structure assignments in YAML
ArtifactStructures:
- Name: Architecture
  Path: Software.Components
  Values:
    - Name: SDOAC SW
      Description: SDOA Car Software
      Values:
        - Name: Integration
          Description: Integration Model
        - Name: Control
          Description: Control Model
- Name: Responsibility
  Path: Devisions.Teams
Artifacts:
- Name: EV3Control
  Structures:
  - Path: Software.Components
    Value: SDOAC SW.Integration
  - Path: Devisions.Teams
    Value: Development.Team F
- Name: GlobalPosition
  Description: Subsystem for the position of the vehicle
  Structures:
  - Path: Software.Components
    Value: SDOAC SW.Control
  - Path: Devisions.Teams
    Value: Development.Team P

In Excel, the artifact structure configuration is done within the ArtifactStructure sheet (as part of the artifact configuration, see Artifacts).

When using Excel as configuration format:

  • Only one hierarchy can be defined.

  • Multiple structure levels are allowed.

  • Structure levels have to be defined as columns with a column name beginning with “Structure” (e.g. ‘StructureSoftware’ for a structure level called “Software”).

  • The order of the levels is defined by the order of the columns in the Excel (e.g. Software > Teams > Components).

../../_images/MQC_ContextCategories_ConfigProjectStructure.png

Figure 4.15 Defining artifact structures and artifacts with structure assignments in Excel

4.3.2. Quality Model

MQC provides data source specific quality model configurations. These predefined source files can be directly chosen via the “Suggestions” tab within the Add dialog (see Import / Reload).

If no quality model was imported, MQC recognizes the used data sources (either based on the enabled adapters or from the data currently imported) and loads the corresponding quality model suggestions.

4.3.2.1. Base Measures

The base measures configured in the quality model define the set of expected data. When configuring base measures, MQC expects the following information:

  • DataSourceName

    The tool/source providing a data report containing data to be imported by MQC. Examples are MXAM and TPT.

  • MeasurementName

    A default measurement name, which is used if the measurement can’t be extracted from a report.

    Typically, a measurement means the set of operations done to determine the value for a measure. For example this can be the name of a specific guideline document, which is used for the static analysis of a model, or the kind of test environment like MiL, SiL or PiL.

  • BaseMeasureName

    The name for a group of measures, e.g. “FindingCount” for the data source MXAM.

  • VariableName

    The name of a specific measure inside a measure group, e.g. “Passed” or “Failed”. Full measure names would then be “FindingCount.Passed” or “FindingCount.Failed”.

  • DefaultValue

    The value, which should be set if no value for a specific measure can be extracted from a report, typically 0. (see Default Values)

4.3.2.1.1. Default Values

MQC uses the configured defaults, if the report contains values for at least one other measure belonging to the same base measure group. Otherwise, the whole base measure group is treated as “Missing”.

Listing 4.5 Defining expected base measures with there default values in YAML
BaseMeasures:
- Name: MXAM.GuidelineAnalysis.FindingCount.Aborted
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Canceled
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Failed
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Info
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Passed
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Repaired
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Review
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Unrepaired
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Warning
  DefaultValue: 0
- Name: MXAM.GuidelineAnalysis.FindingCount.Ignored
  DefaultValue: 0

In Excel, the base measure configuration is done in a separate sheet called BaseMeasure.

../../_images/MQC_ConfigQualityModel_BaseMeasures_Import.png

Figure 4.16 Defining expected base measures with there default values in Excel

4.3.2.2. Derived Measures

A fully qualified derived measure consists of:

  • DataSourceName

    Optional, typically the name of the data source of the base measures used to calculate the derived measure.

  • MeasurementName

    Optional, typically the name of the measurement of the base measures used to calculate the derived measure.

  • DerivedMeasureName

    Mandatory, the name for a group of derived measures.

  • VariableName

    Mandatory, the name of a specific derived measure inside a derived measure group.

  • MeasureFunction

    Mandatory, the expression to calculate a derived measure using base measures and/or other derived measures.

    Base and derived measures must be set in square brackets [ and ]!

If no data source and measurement names are configured (short notation), MQC takes data source and measurement from the base and/or derived measures used to calculate the current derived measure.

This means, if the derived measure configuration is in short notation and the base measures used to calculate the derived measure are provided for multiple measurements (e.g. TestCount.Total for MiL and for SiL), the derived measure will be calculated for all the measurements as well.

To calculate a derived measure for a specific measurement only, fully qualified measure names have to be used inside the measure function.

Listing 4.6 Defining derived measures using a short notation in YAML
DerivedMeasures:
- Name: GuidelinesCalc.Passed
  Expression: '[GuidelineCount.Passed]+log(1+[FindingCount.Passed],2)'
- Name: GuidelinesCalc.Repaired
  Expression: '[GuidelineCount.Repaired]+log(1+[FindingCount.Repaired],2)'
- Name: GuidelinesCalc.Unrepaired
  Expression: '[GuidelineCount.Unrepaired]+log(1+[FindingCount.Unrepaired],2)'
- Name: GuidelinesCalc.Failed
  Expression: '[GuidelineCount.Failed]+log(1+[FindingCount.Failed],2)'
- Name: GuidelinesCalc.Info
  Expression: '[GuidelineCount.Passed with Infos]+log(1+[FindingCount.Info],2)'
- Name: GuidelinesCalc.Review
  Expression: '[GuidelineCount.Review]+log(1+[FindingCount.Review],2)'
- Name: GuidelinesCalc.Warning
  Expression: '[GuidelineCount.Warning]+log(1+[FindingCount.Warning],2)'
- Name: GuidelinesCalc.AbortedCanceled
  Expression: '[GuidelineCount.Aborted]+[GuidelineCount.Canceled]'

In Excel, the derived measure configuration is done in a separate sheet called DerivedMeasure.

../../_images/MQC_ConfigQualityModel_DerivedMeasures_Import.png

Figure 4.17 Defining derived measures using a short notation in Excel

4.3.2.3. Quality Properties

When configuring quality properties, MQC expects the following information:

  • QualityPropertyName

    Mandatory, the name of the quality property.

  • Weight

    Optional, a weight to define that quality properties may have more or less influence than others. If nothing is set, a weight of 1 is used.

  • MeasurementFunction

    Mandatory, the expression to calculate a quality property measure value using base measures and/or other derived measures.

    Base and derived measures must be set in square brackets [ and ]!

  • Description

    Optional, an explanation of the quality property may be added to document and visualize the background concept.

If the measurement function is defined in short notation and the base and derived measures used to calculate the quality property are provided for multiple measurements (e.g. TestCount.Total for MiL and for SiL), the quality property will be calculated for all the measurements as well.

To calculate a quality property for a specific measurement only, fully qualified measure names have to be used inside the measurement function.

Listing 4.7 Quality property definition in YAML
QualityProperties:
- Name: Testable Requirements with Test Sequences
  Weight: 2
  Expression: [Testable Requirements with Test Sequences.Reached] /
              [Testable Requirements with Test Sequences.Total]
  Description: >
     Show the percentage of the reached testable requirements
     with test Sequences

In Excel, the quality property configuration is done within the QualityModel sheet (together with the quality structure configuration, see Quality Structures).

../../_images/MQC_ConfigQualityModel_QualityProperties_Import.png

Figure 4.18 Quality property definition in Excel

4.3.2.4. Quality Structures

To group quality properties in order to define different quality aspects, quality structures can be defined.

Quality properties can be assigned to a specific path of such a defined structure.

When using YAML (or JSON) as configuration format:

  • Multiple hierarchies can defined.

  • Each quality property can be assigned to one or multiple hierarchies.

  • The hierarchies can have a different amount of levels.

  • A description can be added also to structure levels, not only to quality properties.

Listing 4.8 Definition of a quality model structure and assigning quality properties to this structure in YAML
QualityStructures:
  - Name: Functional Suitability
    Path: Models.Characteristics.Subcharacteristics
    Values:
      - Name: Product Quality Model
        Values:
          - Name: Correctness
            Description: >
               Grouping the quality properties.

               Quality properties in this group evaluate the successful
               operation of the models.
            Values:
              - Name: Model Design
              - Name: Functional Requirements
                Description: >
                   Grouping the quality properties that evaluate the services,
                   the models must offer
              - Name: Test Sequences
              - Name: Assessments
          - Name: Completeness
            Values:
              - Name: Functional Requirements
              - Name: Test Sequences
              - Name: Assessments
              - Name: Model Coverage
              - Name: Code Coverage
  - Name: Quality Assurance Methods
    Path: Quality Assurance.QA Method.QA Work Product
    Values:
      - Name: Product Quality Model
        Values:
          - Name: Static Analysis
            Values:
              - Name: Guidelines Model Design
              - Name: Guidelines Model Architecture
          - Name: Test
            Values:
              - Name: Requirements
              - Name: Test Sequences
              - Name: Assessments
              - Name: Structural Coverage
QualityProperties:
  - Name: Requirements Compliance
    Weight: 3
    Expression: '[Requirements Compliance.Reached] / [Requirements Compliance.Total]'
    Structures:
      - Path: Models.Characteristics.Subcharacteristics
        Value: Product Quality Model.Correctness.Functional Requirements
      - Path: Quality Assurance.QA Method.QA Work Product
        Value: Product Quality Model.Test.Requirements

In Excel, the quality structure configuration is done within the QualityModel sheet.

When using Excel as configuration format:

  • Only one hierarchy can be defined.

  • Multiple structure levels are allowed.

  • Structure levels have to be defined as columns with a column name beginning with “Quality” (e.g. ‘QualityCharacteristics’ for a structure level called “Characteristics”).

  • The order of the levels is defined by the order of the columns in the Excel (e.g. Models > Characteristics > Subcharacteristics).

../../_images/MQC_ConfigQualityModel_QualityModel_Import.png

Figure 4.19 Definition of a quality model structure and assigning quality properties to this structure in Excel

4.3.2.5. Context Categories

For details about defining context categories see Configuration of Context Categories.

4.3.2.6. Actions

For details about actions, see Actions.

Listing 4.9 Action definition in YAML
QualityProperties:
- Name: Testable Requirements with Test Sequences
  Weight: 2
  Expression: [Testable Requirements with Test Sequences.Reached] /
              [Testable Requirements with Test Sequences.Total]
  Actions:
  - Link test sequences to (covered) requirements
  - Derive further test sequences from uncovered requirements

In Excel, the action configuration is done in a separate sheet called ActionList.

../../_images/MQC_ConfigQualityModel_Actions_Import.png

Figure 4.20 Defining recommended actions for improving quality properties in Excel

4.3.3. Context Categories

Details about how to switch on or off categories in MQC can be found in Context Categories.

4.3.3.1. Configuration of Context Categories

Context categories have to be configured in the quality model (see Quality Model).

A context category definition may contain:

  • name of the context category

    mandatory, later used to be assigned to artifacts

  • expected data

    optional, white list

  • excluded data

    optional, black list

Both, expected and excluded data has to be defined using comma-separated lists of data sources, measurements and/or measures. Use either the full name or name patterns (including wildcards).

Listing 4.10 Configuring context categories in YAML
ContextCategories:
- Name: StaticAnalysisOnly
  Expect: MXAM, MXRAY
- Name: TestNoCoverage
  Expect: MXAM, MXRAY, MTest
  Exclude: MTest.*.Model*Coverage

In Excel, the context category configuration is done in a separate sheet called ContextCategory.

../../_images/MQC_ContextCategories_ConfigQualityModel.png

Figure 4.21 Configuring context categories in Excel

An empty “Expect” field means, all data is expected for that context category. To state that no data is expected at all, use the key word “none”.

An empty “Exclude” field means, nothing is excluded, but still, the amount of data may be reduced by specifying expected data.

4.3.3.2. Assign Context Categories to Artifacts

Context categories can be assigned to one or multiple artifacts in the project structure configuration source (see Artifacts).

Listing 4.11 Assign context categories to artifacts in YAML
 Artifacts:
   - Name: EV3Control_main
   - Name: ObstacleDetection
     ContextCategories:
       - MLC_CC
   - Name: GlobalPosition
     ContextCategories:
       - Global_CC
   - Name: ManageVehicleStates
     ContextCategories:
       - MLC_CC

In Excel, the assignment of context categories to artifacts is done within the ArtifactStructure sheet.

../../_images/MQC_ContextCategories_ConfigProjectStructure.png

Figure 4.22 Assign context categories to artifacts in Excel

When no context category is assigned, i.e. the field is left empty, all available data is shown for that artifact.

4.3.4. Target Values

To configure target values in MQC, the following information is necessary:

  • ArtifactName

    The name of the artifact a target value shall be applied to.

  • MilestoneName

    The name of the milestone a target value shall be reached at.

  • MeasureName[.TargetName]

    The name of the measure, data or quality property, a target value shall be applied to, which is optionally followed by a specific target value name:

    • Targets for data measures (fully qualified):

      DataSourceName.MeasurementName.BaseMeasureName.VariableName[.TargetName]

    • Targets for quality properties:

      QualityPropertyName[.TargetName]

  • TargetValue

    The value, e.g. a percentage value for quality properties, that is expected to be reached at the date of the configured milestone.

If the configured measure name does not contain a target name at the end, a default target name “Target” is applied by MQC instead.

It is not necessary to define a target for each configured milestone (see Calculation of Target Values per Revision).

Listing 4.12 Defining target values per measure, per artifact and per milestone in YAML
Artifacts:
- Name: ManageVehicleStates
  Milestones:
  - Name: Model Unit Start
    Targets:
    - Name: Guideline Compliance (GuidelineAnalysis)
      Value: 80
    - Name: Requirements Compliance
      Value: 60
  - Name: Model Unit Extension
    Targets:
    - Name: Guideline Compliance (GuidelineAnalysis)
      Value: 80
    - Name: Requirements Compliance
      Value: 80
- Name: EV3Control_main
  Milestones:
  - Name: Model Unit Start
    Targets:
    - Name: Guideline Compliance (GuidelineAnalysis)
      Value: 0
      - Name: Requirements Compliance
        Value: 0
    - Name: Requirements with Reviewed Testability
      Value: 0

In Excel, the target value configuration is done in a sheet called Target Values.

../../_images/MQC_TargetValues_Configuration.png

Figure 4.23 Defining target values per measure, per artifact and per milestone in Excel

Details about how to switch on or off targets in MQC can be found in Target values in visualizations (Custom pages).

4.3.5. Annotations

For a proper description of necessary configuration parameters, refer to Annotations.

Mandatory parameters are:

  • Title

  • Artifact

  • Quality Property

Listing 4.13 Defining annotation source in YAML
Authors:
- Model Engineering Solutions GmbH
Annotations:
- Title: 80% because halted for technical reasons
  Author: mqc_admin
  Artifact: EV3Control_main
  QualityProperty: Code Condition Coverage
  ValidFrom: 2021-10-11
  ValidTo: 2021-10-17
  ConditionType: Bin
  ConditionBins:
    - Acceptable
  TargetType: Quality
  TargetValue: 80.00
- Title: Bad because not enough tests
  Author: mqc_admin
  Artifact: GlobalPosition
  QualityProperty: Testable Requirements with Test Sequences
  ValidFrom: 2021-10-11
  ValidTo: 2021-10-17
  ConditionType: Bin
  ConditionBins:
    - Good
  TargetType: Bin
  TargetValue: Bad
- Title: Because Model Coverage - Decision under 80%
  Description: see Data Origins
  Author: mqc_admin
  Artifact: ObstacleDetection
  QualityProperty: Model Decision Coverage
  ValidFrom: 2021-10-11
  ValidTo: 2021-10-17
  ConditionType: Bin
  ConditionBins:
    - Acceptable
  TargetType: Quality
  TargetValue: 75.00

All other configurations are optional and can be added respectively edited later directly in MQC (see Annotations).