7. Data Sources

This chapter gives you an overview about what Data Sources MQC supports by means of its base adapters. Therefore, the user is able to keep track on those measures that are imported into MQC and can be used for the definition of the Quality Model.

The current release of MQC supports the following tools: MES Model Examiner® (MXAM), MES M-XRAY® (MXRAY) and MES Test Manager® (MTest), as well as PikeTec TPT, Razorcat Tessy, MathWorks Polyspace, BTC EmbeddedTester, Verifysoft Testwell CTC++, Danlawinc MxSuite, and QA-System QAC.

For the case there is no adapter, MQC puts at the user’s disposal the possibility of a manual import of Base Measures (see Manual data import for unsupported data sources (Excel Template)).

7.1. MES Model Examiner® (MXAM)

7.1.1. MXAM Data imported by MQC

MQC supports the standard MXAM report file (.mxmr).

The following example of an MXMR report describes which information is imported by MQC’s MXAM adapter to MQC:

  • from the <RReport> element, specifically the date element:

    • ReportDateTime

  • from the <artifacts> element, the <properties key="modelname"> element:

    • ArtifactName

MXAM provides guideline and finding result data for each artifact in the MXMR Report. Therefore, each artifact section will be parsed to get the information of the artifact result, model type and all findings and guideline results.

  <artifacts result="Review" adapterId="com.modelengineers.mxam.tooladapter.matlab" storageNature="Tool artifact">
  <properties key="modelname" value="ReportReviewSldemo_wheelspeed_absbrake" visible="true"/>
  <properties key="modeltype" value="MOD" visible="true"/>
  <properties key="dynamic" value="" visible="true"/>
  …
  <summary itemType="Findings">
    <statistic resultType="Review" count="3"/>
    <statistic resultType="Failed" count="66"/>
    <statistic resultType="Info" count="28"/>
    <statistic resultType="Passed" count="12"/>
    <statistic resultType="Ignored" count="2"/>
  </summary>
  …
  <summaries itemType="Guidelines">
    <statistic resultType="Review" count="1"/>
    <statistic resultType="Failed" count="5"/>
    <statistic resultType="Passed with Infos" count="24"/>
    <statistic resultType="Passed" count="11"/>
  </summaries>
</artifacts>

The adapter reads those measures to be found in the Findings and Guidelines header: <Summary itemType="Findings"> and <Summary itemType="Guidelines">. Please, note that a MXAM report can contain various artifacts and for each artifact MQC reads out the Findings and Guidelines Summary, that are saved as FindingCount and GuidelineCount, respectively:

  • Review

  • Failed

  • Info (for FindingCount) and Passed with Infos (for GuidelineCount)

  • Passed

  • Ignored

  • Aborted

  • Canceled

  • Repaired

  • Unrepaired

  • Warning (for FindingCount) and Warnings (for GuidelineCount)

To get more insight of how these Base Measures are processed to a measurement function, please refer to Quality Model for MXAM.

7.2. MES M-XRAY® (MXRAY)

7.2.1. MXRAY Data imported by MQC

MQC supports the Standard XML MXRAY Report file.

The following information is extracted by the MQC/ MXRAY adapter:

  • from the <Timestamp> element

    • ReportDateTime

  • from the <SubsystemQualityOverview> header all included elements, usually these are:

    • Local Complexity

    • Level

    • %Elementary Inputs Unused (globally)

    • Cyclomatic Complexity

    • Inports

    • Outports

    For each of these Measures the variables Good, Acceptable and Bad are imported. Furthermore for Local Complexity the variables LowerBoundOfAcceptable and LowerBoundOfBad are read .

  • from the GlobalValueSummary element

    • Global Complexity (Ref0)

    • Global Complexity (Ref1)

    • Global Complexity (RefN)

  • from the CloneGroups element

    • NumberOfDetectedCloneGroups

    • NumberOfSubsystemsAnalyzed

    • NumberOfUniqueSubsystemsInAllCloneGroups

    • NumberOfSubsystemsInAllCloneGroups

Note

In MQC CloneGroups.NumberOfUniqueSubsystemsInAllCloneGroups is shown as CloneGroups.Bad:

Bad = NumberOfUniqueSubsystemsInAllCloneGroups

Additionally CloneGroups.NumberOfSubsystemsAnalyzed is used together with CloneGroups.NumberOfUniqueSubsystemsInAllCloneGroups to calculate a value for CloneGroups.Good:

Good = NumberOfSubsystemsAnalyzed - NumberOfUniqueSubsystemsInAllCloneGroups

To get more insight of how these Base Measures are processed to a measurement function, please refer to Quality Model for M-XRAY.

7.3. MES Test Manager® (MTest)

7.3.1. MTest Data imported by MQC

The MQC-MTest adapter supports the MQC-XML format for MTest Report files.

The MTest XML report consists of one <DataEntryList> header, which contains several <DataEntry> elements, each of them containing all the information for one Base Measure and Artifact.

The following information is extracted by the MQC/MTest adapter:

  • from the <RevisionDate> element

    • ReportDateTime

  • from the <ArtifactNameOrAlias> element

    • ArtifactName

  • from the <DataSourceNameOrAlias> element the BaseMeasureName with its two variables (read out of the <DataSourceValue> element):

    • Absolute stored in MQC as Reached

    • Reference stored in MQC as Total

According to this pattern, from the <DataEntryList> element, the following Base Measures are imported with its respective variables (Reached and Total):

  • Assessment Work Progress

  • Model Condition Coverage

  • Model Decision Coverage

  • Requirements Compliance

  • Requirements with Reviewed Testability

  • Reviewed Assessments

  • Reviewed Test Sequences

  • Test Sequence Work Progress

  • Test Sequences Compliance

  • Testable Requirements with Assessments

  • Testable Requirements with Test Sequences

  • Testable Requirements

For the following Base Measures only exists one value that is stored in the variable Reached:

  • Total Assessments

  • Total Requirements

  • Total Test Sequences

To get more insight of how these Base Measures are processed to a measurement function, please refer to Quality Model for MTest.

7.4. PikeTec TPT

TPT is considered an (xml-based) external test tool for which an MQC adapter executes the following operations presented by the example of the following extract of an TPT example XML report:

<Header ExecutionConfig="Lights Control MATLAB" ExecutionDate="14:47:58 10.05.2016" TptFileName="D:\requirements.tpt" TptVersion="8u2">
  <Property Name="Model Under Test" Value="D:\matlab-platform\lights_control_simulink.mdl"/>
  <Property Name="System Under Test" Value="lights_control_simulink/lights_control"/>
  <Platform History="100" Name="MATLAB-Platform" Stepsize="10000" Timeout="60000000">
    <Property Name="MATLAB Version" Value="MATLAB 8.4"/>
  </Platform>
</Header>
…
<Summary AssessmentDuration="2.518" ExecutionDuration="2.078">
  <ExecutionSummary Errors="0" Failed="6" Inconclusive="0" Succeeded="5" Tests="11"/>
</Summary>

MQC reads out

  • from the main <Header.. >

    • ExecutionDateTime (stored in MQC as ReportDateTime)

    • SystemUnderTest Value (stored in MQC as ArtifactName), in this case “lights_control_simulink/lights_control”. Please, note that TPT stores in the XML only the name (instead of the complete path) of the subsystem

    • TptVersion

    • PlatFormName

  • from the <ExecutionSummary> header the categories

    • Tests (stored in MQC as TestCount.Test)

    • Succeeded (stored in MQC as TestCount.Succeeded)

    • Failed (stored in MQC as TestCount.Failed)

    • Errors (stored in MQC as TestCount.Errors)

    • Inconclusive (stored in MQC as TestCount.Inconclusive)

MQC offers an additional structuring method, called Measurement described in Section 7.11. Herewith, different Measurements with the same BaseMeasure and Variable name can be read out. You can assign the measurement name of MiL, SiL or PiL, which can be stated within the file name or the PlatFormName with the following syntax _SIL_.

7.5. Razorcat Tessy

From the extract of the Tessy Example XML report,

<report success="notok" tessy_version="4.0.15" xml_version="3">
<statistic notexecuted="0" notok="11" ok="54" total="65">
  <category count="54" name="ok"/>
  <category count="11" name="notok"/>
  <category count="0" name="notexecuted"/>
</statistic>
<info date="2018-08-23" time="16:20:30+0200"/>
…
<tessyobject id="1024" level="0" name="Testsuite" success="notok" type="project">

MQC reads

  • from the main (report) header:

    • tessy_version (stored in MQC as TessyReportVersion)

  • from the <statistics..> header the categories:

    • ok

    • notok

    • notexecuted.

  • from the <info..> header:

    • date and time (stored in MQC as ReportDateTime)

  • from the <tessyobject..> header:

    • name (stored in MQC as ArtifactName)

    • type (stored in MQC as TessyObjectType)

The Tessy data is imported to the transformed ValueFact table maintaining the same notation of the Tessy classification of ok, notok and notexecuted, yet assigning them to the BaseMeasure TestCount.

7.6. MathWorks Polyspace

7.6.1. Polyspace Xml-adapter

If a Polyspace Xml report is created, MQC reads out from the xml-file:

  • PubDateTime (stored in MQC as ReportDateTime)

  • Subtitle (stored in MQC as ArtifactName)

  • PolyspaceVersion

  • MISRA-C Checker.Violations

  • MISRA-C Checker.Pass_Fail

This xml-file refers to several xfrag-files in the Polyspace-doc directory to be found on the same level as the xml-file.

From the image-000-chapter.xfrag-file,

<title>Run-Time Checks Summary</title>
  <tgroup align="left" cols="2"><colspec colname="1" colnum="1" colwidth="600*"/><colspec colname="2" colnum="2" colwidth="600*"/><thead>
  <row bgcolor="#6A80BF">
    <entry>Run-Time Checks</entry><entry>Enabled</entry><?dbfo bgcolor="#6A80BF"?></row></thead>
  <tbody>
  <row bgcolor="#F0F2F9">
    <entry>Number of Red Checks</entry><entry>
    <emphasis role="red">0</emphasis></entry>
    <?dbfo bgcolor="#F0F2F9"?>
  </row>
  <row bgcolor="#FAFBFE">
    <entry>Number of Gray Checks</entry>
    <entry><emphasis role="gray">0</emphasis></entry><?dbfo bgcolor="#FAFBFE"?>
  </row>
  <row bgcolor="#F0F2F9">
    <entry>Number of Orange Checks</entry>
    <entry><emphasis role="orange">0</emphasis></entry>
    <?dbfo bgcolor="#F0F2F9"?></row><row bgcolor="#FAFBFE">
    <entry>Number of Green Checks</entry>
    <entry><emphasis role="green">220</emphasis></entry><?dbfo bgcolor="#FAFBFE"?>
  </row>
  <row bgcolor="#F0F2F9">
    <entry>Proven</entry>
    <entry>100.0%</entry><?dbfo bgcolor="#F0F2F9"?>
  </row>
  <row bgcolor="#FAFBFE"><entry>Pass/Fail</entry>
  <entry>-</entry><?dbfo bgcolor="#FAFBFE"?></row></tbody>
  </tgroup>
</table>

MQC extracts the following information, modifies and transforms it via predefined data table transformations:

  • Number of Red Checks (stored as Run-Time Checks.Major in MQC)

  • Number of Gray Checks (stored as Run-Time Checks.Minor in MQC)

  • Number of Orange Checks (stored as Run-Time Checks.Moderate in MQC)

  • Number of Green Checks (stored as Run-Time Checks.Good in MQC)

  • Percentage of Proven (stored as Run-Time Checks.Proven in MQC)

  • Pass/Fail (stored as Run-Time Checks.Pass_Fail in MQC)

7.6.2. Polyspace Text-adapter

If a Polyspace tab-separated Text report is created, MQC reads from this txt-file:

  • Last modified time of file (stored in MQC as ReprtDateTime)

  • File

  • Family

  • Color

From the File column MQC extracts the file paths of all files used to create the report. MQC then obtains the common prefix from these file paths and extracts the last directory from the prefix. This is stored as the Artifact Name in MQC. For example if paths read from the “File” column are as follows:

  • E:\D42\A987R\InputOut\Swc_HMK\TLSim\Rte.c

  • E:\D42\A987R\InputOut\Swc_HMK\TLProj\TL_Swc_WIM\Swc_WIM.c

  • E:\D42\A987R\InputOut\Swc_HMK\TLSim\J_GLOBAL_STD.h

The common prefix is marked above and MQC extracts the last common directory Swc_WIM as the artifact name.

MQC extracts from the tab-separated columns Family as base measure and aggregated Color count as variable. MQC then modifies and transforms it via predefind data table transformations as follows:

  • Family:Run-time Check and Color:Red (stored as Run-Time Checks.Major in MQC)

  • Family:Run-Time Check and Color:Gray (stored as Run-Time Checks.Minor in MQC)

  • Famliy:Run-Time Check and Color:Orange (stored as Run-Time Checks.Moderate in MQC)

  • Family:Run-Time Check and Color:Green (stored as Run-Time Checks.Good in MQC)

  • Family:Global Variable and Color:Gray (stored as Global Variable.Unused variable in MQC)

  • Family:Global Variable and Color:Not Applicable (stored as Global Variable.Used non-shred variable in MQC)

  • Family:MISRA-C (stored as MISRA-C Checker.Violations in MQC)

7.7. BTC EmbeddedTester

MQC supports two types of Embedded Tester (ET) report formats:

7.7.1. Standard XML report format

If a standard XML report is created, MQC will read from the XML file:

<BTCXmlReport>
  <ProfileInfo lastArchitectureUpdate="Wed Nov 07 16:32:55 CET 2018" lastB2BTestExecution="Sat Aug 11 08:44:00 CEST 2018" lastRBTSILTestExecution="Wed Nov 07 16:39:43 CET 2018" lasteModifier="mes" modelName="TestObject.slx" modelVersion="0.0.1+STD" profileName="UT_TestObject.epp"/>
  <RBT>
    <Tests sumSILErrorExecutions="0" sumSILFailedExecutions="0" sumSILMissingExecutions="0" sumSILOutdatedExecutions="0" sumSILPassedExecutions="7" sumTestCases="7"/>
    <Requirements percentageSILPassedRequirements="100.0" sumRequirements="11" sumSILFailedRequirements="0" sumSILMissingOrOutdatedStatusRequirements="0" sumSILPassedRequirements="11"/>
    <SILFailedRequirements/>
    <RBTCoverageOverview>
      <Statement covered="77.33" handled="78.95" unknown="21.05" unreachable="1.62"/>
      <Decision covered="62.82" handled="65.38" unknown="34.62" unreachable="2.56"/>
      <MCDC covered="66.28" handled="68.6" unknown="31.4" unreachable="2.33"/>
    </RBTCoverageOverview>
  </RBT>
  <B2B>
    <Tests lastB2BTestName="TL MIL vs SIL" status="FAILED_ACCEPTED" sumErrorTests="0" sumFailedAcceptedTests="32" sumFailedTests="0" sumPassedTests="25" sumTests="57"/>
    <B2BCoverageOverview>
      <Statement covered="98.38" handled="100.0" unknown="0.0" unreachable="1.62"/>
      <Decision covered="97.44" handled="100.0" unknown="0.0" unreachable="2.56"/>
      <MCDC covered="97.67" handled="100.0" unknown="0.0" unreachable="2.33"/>
    </B2BCoverageOverview>
  </B2B>
</BTCXmlReport>

MQC extracts the following information, stores and transforms it to the MQC data structure, so that the imported data can be added to the ValueFact table as rows:

  • from the <ProfileInfo> header:

    • lastRBTSILTestExecution (stored in MQC as ReportDateTime)

    • modelName (stored in MQC as ArtifactName)

  • from the <RBTCoverageOverview> header:

    • MCDC covered (stored in MQC as Requirement Based Testing.MCDC)

  • from the <RBT> header:

    • percentageSILPassedRequirements (stored in MQC as Requirement Based Testing.Test coverage)

  • from the <BTB> header:

    • sumTests (stored in MQC as Structural Testing.Nb Total)

    • sumPassedTests (stored in MQC as Structural Testing.Passed)

    • sumFailedAcceptedTests (stored in MQC as Structural Testing.Failed Accepted)

    • sumFailedTests (stored in MQC as Structural Testing.Failed)

    • sumErrorTests (stored in MQC as Structural Testing.Error)

    • MCDC covered (stored in MQC as Structural Testing.MCDC)

7.7.2. MQC XML report format

If the MQC XML report is created, MQC is able to automatically import a great number of Base Measures:

  • Back-2-Back-Results with the following variables:

    • Errors

    • Failed

    • Failed (Accepted)

    • Passed

    • Total Vectors

  • Code Coverage - Condition Coverage with the following variables:

    • Covered

    • Handled

    • Tests

    • Unreachable (n/inf)

  • Code Coverage - Decision/Branch Coverage with the following variables:

    • Covered

    • Handled

    • Tests

    • Unreachable (n/inf)

  • Code Coverage - Modified Condition/Decision Coverage with the following variables:

    • Covered

    • Handled

    • Tests

    • Unreachable (n/inf)

  • Code Coverage - Statement Coverage with the following variables:

    • Covered

    • Handled

    • Tests

    • Unreachable (n/inf)

  • Requirements Coverage with the following variables:

    • Requirements

    • Requirements Covered

    • Requirements Fulfillment

  • Test Execution Results with the following variables:

    • Errors

    • Failed

    • Failed (Accepted)

    • Passed

    • Total Vectors

7.8. Verifysoft Testwell CTC++

MQC supports two types of Verifysoft Testwell CTC++ report formats:

7.8.1. Standard XML report format

MQC will read from the XML file:

<ctc_xml_report>
  <header_info>
    <ctcpost_version>8.0.1</ctcpost_version>
    <copyright>Copyright (c) 1993-2013 Testwell Oy</copyright>
    <copyright>Copyright (c) 2013-2016 Verifysoft Technology GmbH</copyright>
    <report_generated>Fri Jun 05 15:22:09 2020</report_generated>
  </header_info>
  <file name="D:\source\Application\DeviceLayer\test_object.cpp">
    <file_type>source</file_type>
    <instrumentation_mode>multicondition</instrumentation_mode>
    <instrumentation_timestamp>Fri Jun 05 15:21:42 2020</instrumentation_timestamp>
    <sym_rewrite_count>0</sym_rewrite_count>
    <sym_update_count>0</sym_update_count>
    <data_rewrite_count>0</data_rewrite_count>
    <data_update_count>0</data_update_count>
    <file_summary>
      <functions>77</functions>
      <lines>200</lines>
      <measurement_points>16</measurement_points>
      <ter>68</ter>
      <hits>343</hits>
      <all>505</all>
      <statement_ter>92</statement_ter>
      <statement_hits>5310</statement_hits>
      <statement_all>5752</statement_all>
      <statement_na_functions>0</statement_na_functions>
    </file_summary>
  </file>
</ctc_xml_report>

MQC extracts the following information, stores and transforms it to the MQC data structure, so that the imported data can be added to the ValueFact table as rows:

  • from the <header_info> header:

    • <report_generated> (stored in MQC as ReportDateTime)

  • from each <file> header:

    • name (stored in MQC as ArtifactName)

    • from the <file_summary> header:

      • <lines> (stored in MQC as Source lines.count)

      • <measurement_points> (stored in MQC as Measurement points.count)

      • when both <xxx_hits> and <xxx_all> exist

        • <xxx_hits> (stored in MQC as xxx.Reached)

        • <xxx_all> (stored in MQC as xxx.Total)

7.8.2. Standard HTML report format

MQC will read from the HTML file:

<html>
  <head>
    <title>CTC++ Coverage Report - Files Summary</title>
  </head>
  <body>
  <table>
    <tr><td class="info">Symbol file(s)</td><td class="info">:</td><td class="info">c:\temp\Autopilot_ec\Test\Test_autopilot_demo_ec\Autopilot_Mode_Logic\Test001\TSeq001\CTCCov\Data_sil_ec\MON.sym (Thu May 21 05:35:10 2020)</td></tr>
    <tr><td class="info">Listing produced at</td><td class="info">:</td><td class="info">Thu May 21 05:44:18 2020</td></tr>
    <tr><td class="info">Coverage view</td><td class="info">:</td><td class="info">Reduced to decision coverage</td></tr>
    <tr><td class="info">Input listing</td><td class="info">:</td><td class="info">D:\temp\Autopilot_ec\Test\Test_autopilot_demo_ec\Autopilot_Mode_Logic\CTCCov\Data_sil_ec\profile.txt</td></tr>
    <tr><td class="info">HTML generated at</td><td class="info">:</td><td class="info">Thu May 21 05:44:18 2020</td></tr>
    <tr><td class="info">Structural threshold</td><td class="info">:</td><td class="infob">100 %</td></tr>
    <tr><td class="info">Statement threshold</td><td class="info">:</td><td class="infob">100 %</td></tr>
  </table><br>
  <table>
    <thead>
      <tr><th>TER %</th><th>-</th><th colspan="2">decision</th><th>TER %</th><th>-</th><th colspan="2">statement</th><th>File</th></tr>
    </thead>
    <tbody>
      <tr><td class="dirb" colspan="9"><a name="a1"></a>Directory: C:\temp\Autopilot_ec\Test\Test_autopilot_demo_ec\Autopilot_Mode_Logic\Autopilot_Mode_Logic_sil_sil_ec_ert_rtw</td></tr>
      <tr><td class="below">98 %</td><td class="below">-</td><td class="below">(41/42)</td><td width="115"><img border="1" src="red.gif" width="98" height="5" alt=""><img border="1" src="white.gif" width="2" height="5" alt=""></td><td class="below">99 %</td><td class="below">-</td><td class="below">(86/87)</td><td width="115"><img border="1" src="red.gif" width="99" height="5" alt=""><img border="1" src="white.gif" width="1" height="5" alt=""></td><td><a href="indexD1.html" class="underline">Autopilot_Mode_Logic_sil_sil_ec.c</a></td></tr>
      <tr><td class="above">100 %</td><td class="above"></td><td class="above">(0/0)</td><td width="115"><img border="1" src="blue.gif" width="100" height="5" alt=""></td><td class="above">100 %</td><td class="above"></td><td class="above">(0/0)</td><td width="115"><img border="1" src="blue.gif" width="100" height="5" alt=""></td><td><a href="indexD2.html" class="underline">Autopilot_Mode_Logic_sil_sil_ec_data.c</a></td></tr>
      <tr><td class="belowb">98 %</td><td class="below">-</td><td class="below">(41/42)</td><td width="115"><img border="1" src="red.gif" width="98" height="5" alt=""><img border="1" src="white.gif" width="2" height="5" alt=""></td><td class="belowb">99 %</td><td class="below">-</td><td class="below">(86/87)</td><td width="115"><img border="1" src="red.gif" width="99" height="5" alt=""><img border="1" src="white.gif" width="1" height="5" alt=""></td><td class="dirb"><a title="C:\temp\R2018B\Autopilot_ec\Test\Test_autopilot_demo_ec\Autopilot_Mode_Logic\Autopilot_Mode_Logic_sil_sil_ec_ert_rtw">DIRECTORY OVERALL</a></td></tr>
      <tr><td class="ruler" colspan="9">&nbsp;</td></tr>
      <tr><td class="belowb">98 %</td><td class="below">-</td><td class="below">(41/42)</td><td width="115"><img border="1" src="red.gif" width="98" height="5" alt=""><img border="1" src="white.gif" width="2" height="5" alt=""></td><td class="belowb">99 %</td><td class="below">-</td><td class="below">(86/87)</td><td width="115"><img border="1" src="red.gif" width="99" height="5" alt=""><img border="1" src="white.gif" width="1" height="5" alt=""></td><td><a href="indexO.html" class="underlineb">OVERALL</a></td></tr>
    </tbody>
  </table><br>
 </body>
</html>

MQC extracts the following information, stores and transforms it to the MQC data structure, so that the imported data can be added to the ValueFact table as rows:

  • from the first <table> header:

    • read the last <td> of the <tr> with the first <td> equal to ‘Listing produced at’ (stored in MQC as ReportDateTime)

  • from the second <table> header:

    • from <thead> start from third <th> and every three <th> read as MeasureName.

    • from each <tr>:

      • read last <td> as <ArtifactName>

      • from <td> correspondent to <th> read values for that measure in such a way that the first number is fetch as Reached and second as Total variable. (e.g for second row in sample report above, (41/42) is read as decision.Reached = 41 and decision.Total = 42)

      Note

      In reading rows from second table we will ignore <tr> with just one <td> or has class=”dirb” or class=”ruler” or when last <td> is equal to “DIRECTORY OVERALL” or “OVERALL”.

7.9. Danlawinc MxSuite

MQC reads data from the XML file named Report.RegResults.xml:

<MxVDevReportFile>
  <Report>
    <Date>20200423</Date>
    <Time>15:39</Time>
    <ProgramVersion>3.41.1.45984</ProgramVersion>
  </Report>
  <Regression>
    <Overview>
      <Project>
        <Name>CRRCmil</Name>
        <Description>
        </Description>
        <Folder>D:\CITest\CRRCmil</Folder>
        <FileName>CRRCmil.mxp</FileName>
        <ScenarioFolder>.\Regression\Regression-Set1-2020-04-23-15-39-11\ScenariosAndTestCases\</ScenarioFolder>
      </Project>
    </Overview>
    <StatisticsTotals Total="1" PercentTotal="100" Passed="0" PercentPassed="0" Failed="1" PercentFailed="100" Skipped="0" PercentSkipped="0" RunTimeError="0" PercentRunTimeError="0" Missing="0" PercentMissing="0">
      <Testcases Total="1" PercentTotal="100" Passed="0" PercentPassed="0" Failed="1" PercentFailed="100" Skipped="0" PercentSkipped="0" RunTimeError="0" PercentRunTimeError="0" Missing="0" PercentMissing="0" />
    </StatisticsTotals>
  </Regression>
</MxVDevReportFile>

MQC reads out

  • from the <Report>

    • <Date> and <Time> (stored in MQC as ReportDateTime)

  • from each <Regression> header:

    • <Overview><Project><Name> (stored in MQC as ArtiactName)

    • from <StatisticsTotals> header:

      • Total (stored in MQC as Scenarios.Total)

      • Passed (stored in MQC as Scenarios.Passed)

      • Failed (stored in MQC as Scenarios.Failed)

      • Skipped (stored in MQC as Scenarios.Skipped)

      • RunTimeError (stored in MQC as Scenarios.RunTimeError)

      • Missing (stored in MQC as Scenarios.Missing)

    • from <Testcases> header:

      • Total (stored in MQC as Testcases.Total)

      • Passed (stored in MQC as Testcases.Passed)

      • Failed (stored in MQC as Testcases.Failed)

      • Skipped (stored in MQC as Testcases.Skipped)

      • RunTimeError (stored in MQC as Testcases.RunTimeError)

      • Missing (stored in MQC as Testcases.Missing)

7.10. QA-System QAC

MQC supports two types of QA-System QAC report formats:

7.10.1. Standard XML report format

MQC will read from the XML file:

<AnalysisData timestamp="20200423T162554" projectpath="C:/Users/public/AppData/Local/samples/Examples" reportpath="C:/Users/public/AppData/Local/samples/Examples/configs/reports">
  <dataroot type="project">
    <tree type = "rules" >
      <RuleGroup name="xxx" total="5212" active="1112" >
        <Rule id = "" total="5" active="2" text="" >
          <Rule id = "" total="2" active="2" text="" >
            <Message guid = "" total="2" active="2" text="" />
          </Rule>
        </Rule>
        <Rule id = "" total="3" active="0" text="" >
          <Rule id = "" total="3" active="0" text="" >
            <Message guid = "" total="1" active="0" text="" />
            <Message guid = "" total="2" active="0" text="" />
          </Rule>
        </Rule>
      </RuleGroup>
    </tree>
  </dataroot>
  <dataroot type="per-file">
    <File path=".../example.h">
      <tree type="rules">
        <RuleGroup name = "xxx" total="84" active="53" >
          <Rule id="xxx" total="29" active="23" text="There shall be no occurrence of undefined or critical unspecified behaviour" >
            <Message guid = "qac-9.3.1-0602" total="6" active="0" text="" />
            <Message guid = "qac-9.3.1-0603" total="12" active="12" text="" />
            <Message guid = "qac-9.3.1-0836" total="1" active="1" text="" />
            <Message guid = "qac-9.3.1-0848" total="5" active="5" text="" />
            <Message guid = "qac-9.3.1-0854" total="5" active="5" text="" />
          </Rule>
        </RuleGroup>
      </tree>
      <tree type="levels">
        <Level guid = "QA_WARNING" total="55" active="30" name="Warnings" ></Level>
        <Level guid = "QA_ERROR" total="2" active="2" name="Errors" ></Level>
      </tree>
    </File>
  </dataroot>
</AnalysisData>

MQC extracts the following information, stores and transforms it to the MQC data structure, so that the imported data can be added to the ValueFact table as rows:

  • from the <AnalysisData> header:

    • timestamp (stored in MQC as ReportDateTime)

  • from each <File> header inside <dataroot type="per-file">:

    • path (stored in MQC as ArtifactPath)

    • from the first <RuleGroup> inside <tree type="rules">:

      • total (stored in MQC as Violation Count.Total)

      • active (stored in MQC as Violation Count.Active)

    • for each <Level> inside <tree type="levels">:

      • read active if guid = “QA_WARNING” and stored in MQC as Active Diagnostics.WARNING

      • read active if guid = “QA_ERROR” and stored in MQC as Active Diagnostics.ERROR

      • read sum of both above and stored in MQC as Active Diagnostics.Active

    • From <tree type="rules"> header:

      • read number of <Rule> headers that contained just <Message> as child element with active=”0” and stored in MQC as Violated Rules.Violated

      • read number of <Rule> headers that contained just <Message> as child element and stored in MQC as Violated Rules.Total

    • From <tree type="rules"> inside <dataroot type="per-file"> header:

      • read number of <Rule> headers that contained just <Message> as child element with active=”0” and stored in MQC as Project Rules.Violated

      • read number of <Rule> headers that contained just <Message> as child element and stored in MQC as Project Rules.Total

7.10.2. Standard HTML report format

MQC will read from the HTML file:

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body>
    <div id = "head" >
      <div class="stitle">Project &nbsp;&nbsp;: C:/Users/public/AppData/Local/Examples<br/> Status at: 23 Apr, 2019 at 16:25:13</div>
    </div>
    <div class="head">
      <div class="toc">
       <ol>
        <li><a href="#DiagsPerParents">Diagnostics Per Parent Rules</a></li>
        <li><a href="#WorstRules">Most Violated Rules</a></li>
        <li><a href="#FileStatus">File Status</a></li>
        <li><a href="#AnalysisSettings">Analysis Settings</a></li>
        <li><a href="#StatCalc">Calculation Information</a></li>
        <br/>      <br/>      <br/>     </ol>
      </div>
      <div class="summary">
       <table border="0">
        <col border="50"/>
        <tr><td>Number of Files</td><td>185</td></tr>
        <tr><td>Lines of Code (source files only)</td><td>16431</td></tr>
        <tr><td>Total preprocessed code line</td><td>3036</td></tr>
        <tr><td>Diagnostic Count</td><td>948</td></tr>
        <tr><td>Rule Violation Count</td><td>1212</td></tr>
        <tr><td>Violated Rules</td><td>183</td></tr>
        <tr><td>Compliant Rules</td><td>7</td></tr>
        <tr><td>File Compliance Index</td><td>97.41%</td></tr>
        <tr><td>Project Compliance Index</td><td>6.68%</td></tr>
       </table>
      </div>
   </div>
   <div id="content">
     <div class="analstat">
       <table border="1" id="filestat" class="tablesorter">
         <thead>
           <tr><th>Files</th><th>Active Diagnostics</th><th>Violated Rules</th><th>Violation Count</th><th>Compliance Index</th></tr>
         </thead>
         <tbody>
           <tr><td align="left"><a href="..\src\Example.c" title="..\src\Example.c">Example.c</a></td><td align="right">4</td><td align="right">1</td><td align="right">4</td><td align="right">99.42%</td></tr>
         </tbody>
       </table>
     </div>
   </div>
  </body>
</html>

MQC extracts the following information, stores and transforms it to the MQC data structure, so that the imported data can be added to the ValueFact table as rows:

  • from <div class="stitle"> that contains ‘Status at:’ read date and time and stored in MQC as ReportDateTime

  • from <table id="filestat"> header:

    • from <thead> read <th> as MeasureName (just ignore first and last one).

    • from each <tr>:

      • read first <td> as <ArtifactName>

      • from <td> correspondent to <th> read values for that measure.

  • from <table> inside <div class="summary">:

    • read second <td> of <tr> that first is equal to ‘Violated Rules’ (stored in MQC as Project Rules.Violated)

    • read second <td> of <tr> that first is equal to ‘Compliant Rules’ (stored in MQC as Project Rules.Compliant)

7.11. Manual data import for unsupported data sources (Excel Template)

MQC provides the possibility to import data from any other data source using the manual import option.

The manual import has to be done in Excel. You can use this Excel template to create data that should be loaded to MQC. A template to import manual data in MQC, called ManualDataImport_Template.xlsx is provided within: C:\Users\...\AppData\Local\TIBCO\Spotfire\10.3.3\Modules\MES.MQC.CoreExtension_X.X.X.X\Resources\SampleFiles\.

../_images/MQC_DataSources_ManualImport.png

Figure 7.1 Sample file for manual import of two Base Measures for two Artifacts for the same revision

The column Artifact consists of entries representing the objects for which data shall be collected and for which quality shall be computed (e.g. Simulink models, requirements documents, software components).

The entries of the column ReportDateTime are considered by MQC as the days of raising the data, which shall be collected.

Using the syntax DataSource . Measurement . Measure . Variable, you are able to load into your project any data you want. Use DataSource to specify where your data is coming from (e.g. the name of the tool which produced the data). You can optionally fill out Measurement to provide more structuring regarding the data (e.g. the reason why data is collected, the test environment or contexts models have been through, etc). Use the Measure to provide more structuring of your data regarding the result of the measurement. To do that, give a name (the entry of the Measure field) and a Variable. Finally, the Value contains the metric value that you would like to import for the artifact and the related revision. It is recommended to fill out Measurement to achieve a high compliance to ISO 250xx and to avoid that a default name is provided by MQC, which has probably no meaning to you.

As shown in Figure 7.1 you should put your variables using DataSource . Measurement . Measure . Variable syntax within the excel column name. The value for the respective Artifact for a ReportDateTime has to be assigned to the corresponding line.

In case you might want to assign default values for your imported Base Measures, you have to define them in compliance with this syntax within the Quality Model Base Measure excel sheet (see Figure 11.1).

Note

The manual data import supports .xls files, in addition to .xlsx files, and multiple sheets starting from MQC 4.3.