8. Data Source Adapters¶
This chapter gives you an overview about which Data Sources are supported by MQC as base adapters.
The current release of MQC supports the following tools:
For the case there is no adapter, MQC puts at the user’s disposal the possibility of a manual import of Base Measures (see Manual data import for unsupported data sources (Excel Template)).
8.1. MES Model Examiner® (MXAM)¶
8.1.1. MXAM Data imported by MQC¶
MQC supports the standard MXAM report file (.mxmr).
The following example of an MXMR report describes which information is imported by MQC’s MXAM adapter to MQC:
from the
<RReport>
element, specifically thedate
element:ReportDateTime
from the
<artifacts>
element, the<properties key="modelname">
element:ArtifactName
MXAM provides guideline and finding result data for each artifact in the MXMR Report. Therefore, each artifact section will be parsed to get the information of the artifact result, model type and all findings and guideline results.
<artifacts result="Review" adapterId="com.modelengineers.mxam.tooladapter.matlab" storageNature="Tool artifact">
<properties key="modelname" value="ReportReviewSldemo_wheelspeed_absbrake" visible="true"/>
<properties key="modeltype" value="MOD" visible="true"/>
<properties key="dynamic" value="" visible="true"/>
…
<summary itemType="Findings">
<statistic resultType="Review" count="3"/>
<statistic resultType="Failed" count="66"/>
<statistic resultType="Info" count="28"/>
<statistic resultType="Passed" count="12"/>
<statistic resultType="Ignored" count="2"/>
</summary>
…
<summaries itemType="Guidelines">
<statistic resultType="Review" count="1"/>
<statistic resultType="Failed" count="5"/>
<statistic resultType="Passed with Infos" count="24"/>
<statistic resultType="Passed" count="11"/>
</summaries>
</artifacts>
The adapter reads those measures to be found in the Findings and Guidelines
header: <Summary itemType="Findings">
and
<Summary itemType="Guidelines">
. Please note that, an MXAM report can
contain various artifacts and for each artifact MQC reads out the Findings and
Guidelines Summary, that are saved as FindingCount
and GuidelineCount
,
respectively:
Review
Failed
Info
(for FindingCount) andPassed with Infos
(for GuidelineCount)Passed
Ignored
Aborted
Canceled
Repaired
Unrepaired
Warning
(for FindingCount) andWarnings
(for GuidelineCount)
To get more insight of how these Base Measures are processed to a measurement function, please refer to Quality Model for MXAM.
8.2. MES M-XRAY® (MXRAY)¶
8.2.1. MXRAY Data imported by MQC¶
MQC supports the Standard XML MXRAY Report file.
The following information is extracted by the MQC/ MXRAY adapter:
from the
<Timestamp>
elementReportDateTime
from the
<SubsystemQualityOverview>
header all included elements, usually these are:Local Complexity
Level
%Elementary Inputs Unused (globally)
Cyclomatic Complexity
Inports
Outports
For each of these Measures the variables
Good
,Acceptable
andBad
are imported. Furthermore forLocal Complexity
the variablesLowerBoundOfAcceptable
andLowerBoundOfBad
are read .from the
GlobalValueSummary
elementGlobal Complexity (Ref0)
Global Complexity (Ref1)
Global Complexity (RefN)
from the
CloneGroups
elementNumberOfDetectedCloneGroups
NumberOfSubsystemsAnalyzed
NumberOfUniqueSubsystemsInAllCloneGroups
NumberOfSubsystemsInAllCloneGroups
Note
In MQC CloneGroups.NumberOfUniqueSubsystemsInAllCloneGroups
is shown
as CloneGroups.Bad
:
Bad
= NumberOfUniqueSubsystemsInAllCloneGroups
Additionally CloneGroups.NumberOfSubsystemsAnalyzed
is used together
with CloneGroups.NumberOfUniqueSubsystemsInAllCloneGroups
to calculate
a value for CloneGroups.Good
:
Good
= NumberOfSubsystemsAnalyzed
- NumberOfUniqueSubsystemsInAllCloneGroups
To get more insight of how these Base Measures are processed to a measurement function, please refer to Quality Model for M-XRAY.
8.3. MES Test Manager® (MTest)¶
8.3.1. MTest Data imported by MQC¶
The MQC-MTest adapter supports the MQC-XML format for MTest Report files.
The MTest XML report consists of one <DataEntryList>
header, which contains
several <DataEntry>
elements, each of them containing all the information
for one Base Measure and Artifact.
The following information is extracted by the MQC/MTest adapter:
from the
<RevisionDate>
elementReportDateTime
from the
<ArtifactNameOrAlias>
elementArtifactName
from the
<DataSourceNameOrAlias>
element theBaseMeasureName
with its two variables (read out of the<DataSourceValue>
element):Absolute
stored in MQC asReached
Reference
stored in MQC asTotal
According to this pattern, from the <DataEntryList>
element, the following
Base Measures are imported with its respective variables (Reached and Total):
Assessment Work Progress
Model Condition Coverage
Model Decision Coverage
Requirements Compliance
Requirements with Reviewed Testability
Reviewed Assessments
Reviewed Test Sequences
Test Sequence Work Progress
Test Sequences Compliance
Testable Requirements with Assessments
Testable Requirements with Test Sequences
Testable Requirements
For the following Base Measures only exists one value that is stored in the
variable Reached
:
Total Assessments
Total Requirements
Total Test Sequences
To get more insight of how these Base Measures are processed to a measurement function, please refer to Quality Model for MTest.
8.4. PikeTec TPT¶
TPT is considered an (xml-based) external test tool for which an MQC adapter executes the following operations presented by the example of the following extract of an TPT example XML report:
<Header ExecutionConfig="Lights Control MATLAB" ExecutionDate="14:47:58 10.05.2016" TptFileName="D:\requirements.tpt" TptVersion="8u2">
<Property Name="Model Under Test" Value="D:\matlab-platform\lights_control_simulink.mdl"/>
<Property Name="System Under Test" Value="lights_control_simulink/lights_control"/>
<Platform History="100" Name="MATLAB-Platform" Stepsize="10000" Timeout="60000000">
<Property Name="MATLAB Version" Value="MATLAB 8.4"/>
</Platform>
</Header>
…
<Summary AssessmentDuration="2.518" ExecutionDuration="2.078">
<ExecutionSummary Errors="0" Failed="6" Inconclusive="0" Succeeded="5" Tests="11"/>
</Summary>
MQC reads out
from the
<Header.. >
elementExecutionDateTime
(stored in MQC asReportDateTime
)SystemUnderTest Value
(stored in MQC asArtifactName
), in this case “lights_control_simulink/lights_control”. Please, note that TPT stores in the XML only the name (instead of the complete path) of the subsystem
from the
<ExecutionSummary>
element the categoriesTests
(stored in MQC asTestCount.Total
)Succeeded
(stored in MQC asTestCount.Succeeded
)Failed
(stored in MQC asTestCount.Failed
)Errors
(stored in MQC asTestCount.Errors
)Inconclusive
(stored in MQC asTestCount.Inconclusive
)
<Header>
<Property Name="System under Test" Value=""/>
<Property Name="Revision" Value=""/>
<Platform History="100" Name="C Platform" Stepsize="10000" Timeout="60000000">
<Property Name="Platform Mapping" Value="<none>"/>
<CoverageData Coverage="1.0" CoverageType="Function" ToolName="CTC++" ToolVersion=""/>
<CoverageData Coverage="0.29" CoverageType="Statement" ToolName="CTC++" ToolVersion=""/>
<CoverageData Coverage="0.29" CoverageType="Decision" ToolName="CTC++" ToolVersion=""/>
<CoverageData Coverage="0.16" CoverageType="Condition" ToolName="CTC++" ToolVersion=""/>
<CoverageData Coverage="0.21" CoverageType="MC/DC" ToolName="CTC++" ToolVersion=""/>
<CoverageData Coverage="0.18" CoverageType="Multicondition" ToolName="CTC++" ToolVersion=""/>
</Platform>
</Header>
from the
<Header><Platform>
branch all<CoverageData>
elements using the attributeCoverageType
to distinguishFunction Coverage CTC.Ratio
Statement Coverage CTC.Ratio
Decision Coverage CTC.Ratio
Condition Coverage CTC.Ratio
MC/DC Coverage CTC.Ratio
Multicondition Coverage CTC.Ratio
<Requirements>
<Requirement Id="SPEC-12" Text="Functional requirements"/>
<Requirement Id="SPEC-13" State="SUCCESS" Text="If light_switch is ON, then headlight shall immediately be ON."/>
</Requirements>
<Testcases Number="11">
<Testcase Description="..." Requirements="SPEC-6 SPEC-13"/>
</Testcases>
<Assesslets Number="22">
<Group Name="Assesslets">
<Assesslet Id="1" Name="..." Requirements="SPEC-13"/>
</Group>
</Assesslets>
from the
<Requirements>
elementRequirements Count.Total
(total number of<Requirement>
elements)Requirements Count.Testable
(number of requirements with attributeState
)Requirements Status.Passed
(attributeState="SUCCESS"
)Requirements Status.Failed
(attributeState="FAILED"
)Requirements Status.ExecutionError
(attributeState="EXECUTION_ERROR"
)Requirements Status.DontKnow
(attributeState="DONT_KNOW"
)Requirements Status.NotCovered
(attributeState="NOT_COVERED"
)Testable Requirements with Test Cases.Reached
(number of requirements with attributeState
and any reference in element<Testcases>
)Testable Requirements with Assesslets.Reached
(number of requirements with attributeState
and any reference in element<Assesslets>
)
MQC offers an additional structuring element called Measurement
(described
in Section 6.1). Herewith, test results (same
base measure and variable name) may be read for different contexts, e.g. MiL or
SiL.
The measurement name is expected as part of the file name or read from the
Name
attribute of the <Platform>
element. In both cases, MQC expects
the following syntax: _SIL_
or _MIL_
. If not existing, the configured
default measurement name is taken
(see Section 12.1).
8.5. Razorcat Tessy¶
From the extract of the Tessy Example XML report,
<report success="notok" tessy_version="4.0.15" xml_version="3">
<statistic notexecuted="0" notok="11" ok="54" total="65">
<category count="54" name="ok"/>
<category count="11" name="notok"/>
<category count="0" name="notexecuted"/>
</statistic>
<info date="2018-08-23" time="16:20:30+0200"/>
…
<tessyobject id="1024" level="0" name="Testsuite" success="notok" type="project">
MQC reads
from the main (
report
) header:tessy_version
(stored in MQC asTessyReportVersion
)
from the
<statistics..>
header the categories:ok
notok
notexecuted
.
from the
<info..>
header:date
andtime
(stored in MQC asReportDateTime
)
from the
<tessyobject..>
header:name
(stored in MQC asArtifactName
)type
(stored in MQC asTessyObjectType
)
The Tessy data is imported to the transformed ValueFact table maintaining the
same notation of the Tessy classification of ok
, notok
and
notexecuted
, yet assigning them to the BaseMeasure TestCount
.
8.6. MathWorks Polyspace¶
8.6.1. Polyspace Xml-adapter¶
If a Polyspace Xml report is created, MQC reads out from the xml-file:
ReportDateTime
: from elementPubDateTime
ArtifactPath
: from elementSubtitle
ArtifactName
: same asArtifactPath
The xml-file refers to several xfrag-files in the Polyspace-doc
directory
to be found on the same level as the xml-file.
From the image-000-chapter.xfrag
-file, MQC extracts
BaseMeasureName
: from thetitle
elements of the tablesVariableName
: firstentry
element of each table body rowMeasureValue
: secondentry
element of each table body row
<table>
<title>Coding Rules Summary - MISRA-C Checker</title>
<tgroup>
<tbody>
<row><entry>Violations</entry><entry>52</entry></row>
<row><entry>Pass/Fail</entry><entry>-</entry></row>
</tbody>
</tgroup>
</table>
<table>
<title>Run-Time Checks Summary</title>
<tgroup>
<tbody>
<row><entry>Number of Red Checks</entry><entry><emphasis role="red">2</emphasis></entry></row>
<row><entry>Number of Gray Checks</entry><entry><emphasis role="gray">10</emphasis></entry></row>
<row><entry>Number of Orange Checks</entry><entry><emphasis role="orange">13</emphasis></entry></row>
<row><entry>Number of Green Checks</entry><entry><emphasis role="green">205</emphasis></entry></row>
<row><entry>Proven</entry><entry>100.0%</entry></row>
<row><entry>Pass/Fail</entry><entry>-</entry></row>
</tbody>
</tgroup>
</table>
<table>
<title>Global Variable Summary</title>
<tgroup>
<tbody>
<row><entry>Used non-shared variable</entry><entry>51</entry></row>
</tbody>
</tgroup>
</table>
The extracted data is then modified and transformed:
Number of Red Checks
stored asRun-Time Checks.Major
Number of Gray Checks
stored asRun-Time Checks.Minor
Number of Orange Checks
stored asRun-Time Checks.Moderate
Number of Green Checks
stored asRun-Time Checks.Good
Percentage of Proven
stored asRun-Time Checks.Proven
Pass/Fail
stored asRun-Time Checks.Pass_Fail
8.6.2. Polyspace Text-adapter¶
If a Polyspace tab-separated Text report is created, MQC reads from this txt-file:
ReportDateTime
: time stamp of the last modification of the txt-fileArtifactPath
: extracted fromFile
column (see below)ArtifactName
: same asArtifactPath
BaseMeasureName
: extracted fromFamily
columnVariableName
: extracted fromColor
columnMeasureValue
: aggregated count perColor
To get the ArtifactPath
, MQC extracts the file paths of all files used to
create the report from the File
column. MQC then obtains the common prefix
from these file paths and takes the last directory from that prefix. This is
stored as ArtifactPath
.
For example if paths read from the “File” column are as follows:
C:\workspace\W33\162R\02.ModelBasedDesign\WrapperInput\GlobalPosition\TLProj\TL_GlobalPosition\GlobalPosition.h
C:\workspace\W33\162R\02.ModelBasedDesign\WrapperInput\GlobalPosition\TLSim\Rte_GlobalPosition.h
C:\workspace\W33\162R\02.ModelBasedDesign\WrapperInput\GlobalPosition\TLSim\TL_GlobalPosition_fri.h
In this example the common prefix is
C:\workspace\W33\162R\02.ModelBasedDesign\WrapperInput\GlobalPosition
and
MQC extracts the last common directory GlobalPosition
as the artifact path.
After reading the extracted data is then modified and transformed as follows:
If
Family
equalsRun-time Check
:Color
=Red
is stored asRun-Time Checks.Major
Color
=Gray
is stored asRun-Time Checks.Minor
Color
=Orange
is stored asRun-Time Checks.Moderate
Color
=Green
is stored asRun-Time Checks.Good
If
Family
equalsGlobal Variable
:Color
=Gray
is stored asGlobal Variable.Unused variable
Color
=Not Applicable
is stored asGlobal Variable.Used non-shared variable
Family
=MISRA-C
is stored asMISRA-C Checker.Violations
8.7. BTC EmbeddedTester¶
MQC supports two types of Embedded Tester (ET) report formats:
MQC XML report format (previous versions of ET)
8.7.1. Standard XML report format¶
If a standard XML report is created, MQC will read from the XML file:
<BTCXmlReport>
<ProfileInfo lastArchitectureUpdate="Wed Nov 07 16:32:55 CET 2018" lastB2BTestExecution="Sat Aug 11 08:44:00 CEST 2018" lastRBTSILTestExecution="Wed Nov 07 16:39:43 CET 2018" lasteModifier="mes" modelName="TestObject.slx" modelVersion="0.0.1+STD" profileName="UT_TestObject.epp"/>
<RBT>
<Tests sumSILErrorExecutions="0" sumSILFailedExecutions="0" sumSILMissingExecutions="0" sumSILOutdatedExecutions="0" sumSILPassedExecutions="7" sumTestCases="7"/>
<Requirements percentageSILPassedRequirements="100.0" sumRequirements="11" sumSILFailedRequirements="0" sumSILMissingOrOutdatedStatusRequirements="0" sumSILPassedRequirements="11"/>
<SILFailedRequirements/>
<RBTCoverageOverview>
<Statement covered="77.33" handled="78.95" unknown="21.05" unreachable="1.62"/>
<Decision covered="62.82" handled="65.38" unknown="34.62" unreachable="2.56"/>
<MCDC covered="66.28" handled="68.6" unknown="31.4" unreachable="2.33"/>
</RBTCoverageOverview>
</RBT>
<B2B>
<Tests lastB2BTestName="TL MIL vs SIL" status="FAILED_ACCEPTED" sumErrorTests="0" sumFailedAcceptedTests="32" sumFailedTests="0" sumPassedTests="25" sumTests="57"/>
<B2BCoverageOverview>
<Statement covered="98.38" handled="100.0" unknown="0.0" unreachable="1.62"/>
<Decision covered="97.44" handled="100.0" unknown="0.0" unreachable="2.56"/>
<MCDC covered="97.67" handled="100.0" unknown="0.0" unreachable="2.33"/>
</B2BCoverageOverview>
</B2B>
</BTCXmlReport>
MQC extracts the following information, stores and transforms it to the MQC data structure, so that the imported data can be added to the ValueFact table as rows:
from the
<ProfileInfo>
header:lastRBTSILTestExecution
(stored in MQC asReportDateTime
)modelName
(stored in MQC asArtifactName
)
from the
<RBTCoverageOverview>
header:MCDC covered
(stored in MQC asRequirement Based Testing.MCDC
)
from the
<RBT>
header:percentageSILPassedRequirements
(stored in MQC asRequirement Based Testing.Test coverage
)
from the
<BTB>
header:sumTests
(stored in MQC asStructural Testing.Nb Total
)sumPassedTests
(stored in MQC asStructural Testing.Passed
)sumFailedAcceptedTests
(stored in MQC asStructural Testing.Failed Accepted
)sumFailedTests
(stored in MQC asStructural Testing.Failed
)sumErrorTests
(stored in MQC asStructural Testing.Error
)MCDC covered
(stored in MQC asStructural Testing.MCDC
)
8.7.2. MQC XML report format¶
If the MQC XML report is created, MQC is able to automatically import a great number of Base Measures:
Back-2-Back-Results
with the following variables:Errors
Failed
Failed (Accepted)
Passed
Total Vectors
Code Coverage - Condition Coverage
with the following variables:Covered
Handled
Tests
Unreachable (n/inf)
Code Coverage - Decision/Branch Coverage
with the following variables:Covered
Handled
Tests
Unreachable (n/inf)
Code Coverage - Modified Condition/Decision Coverage
with the following variables:Covered
Handled
Tests
Unreachable (n/inf)
Code Coverage - Statement Coverage
with the following variables:Covered
Handled
Tests
Unreachable (n/inf)
Requirements Coverage
with the following variables:Requirements
Requirements Covered
Requirements Fulfillment
Test Execution Results
with the following variables:Errors
Failed
Failed (Accepted)
Passed
Total Vectors
8.8. Verifysoft Testwell CTC++¶
MQC supports two types of Verifysoft Testwell CTC++ report formats:
8.8.1. Standard XML report format¶
MQC will read from the XML file:
<ctc_xml_report>
<header_info>
<ctcpost_version>8.0.1</ctcpost_version>
<copyright>Copyright (c) 1993-2013 Testwell Oy</copyright>
<copyright>Copyright (c) 2013-2016 Verifysoft Technology GmbH</copyright>
<report_generated>Fri Jun 05 15:22:09 2020</report_generated>
</header_info>
<file name="D:\source\Application\DeviceLayer\test_object.cpp">
<file_type>source</file_type>
<instrumentation_mode>multicondition</instrumentation_mode>
<instrumentation_timestamp>Fri Jun 05 15:21:42 2020</instrumentation_timestamp>
<sym_rewrite_count>0</sym_rewrite_count>
<sym_update_count>0</sym_update_count>
<data_rewrite_count>0</data_rewrite_count>
<data_update_count>0</data_update_count>
<file_summary>
<functions>77</functions>
<lines>200</lines>
<measurement_points>16</measurement_points>
<ter>68</ter>
<hits>343</hits>
<all>505</all>
<statement_ter>92</statement_ter>
<statement_hits>5310</statement_hits>
<statement_all>5752</statement_all>
<statement_na_functions>0</statement_na_functions>
</file_summary>
</file>
</ctc_xml_report>
MQC extracts the following information, stores and transforms it to the MQC data structure, so that the imported data can be added to the ValueFact table as rows:
from the
<header_info>
header:<report_generated>
(stored in MQC asReportDateTime
)
from each
<file>
header:name
(stored in MQC asArtifactName
)from the
<file_summary>
header:<lines>
(stored in MQC asSource lines.count
)<measurement_points>
(stored in MQC asMeasurement points.count
)when both
<xxx_hits>
and<xxx_all>
exist<xxx_hits>
(stored in MQC asxxx.Reached
)<xxx_all>
(stored in MQC asxxx.Total
)
8.8.2. Standard HTML report format¶
MQC will read from the HTML file:
<html>
<head>
<title>CTC++ Coverage Report - Files Summary</title>
</head>
<body>
<table>
<tr><td class="info">Symbol file(s)</td><td class="info">:</td><td class="info">c:\temp\Autopilot_ec\Test\Test_autopilot_demo_ec\Autopilot_Mode_Logic\Test001\TSeq001\CTCCov\Data_sil_ec\MON.sym (Thu May 21 05:35:10 2020)</td></tr>
<tr><td class="info">Listing produced at</td><td class="info">:</td><td class="info">Thu May 21 05:44:18 2020</td></tr>
<tr><td class="info">Coverage view</td><td class="info">:</td><td class="info">Reduced to decision coverage</td></tr>
<tr><td class="info">Input listing</td><td class="info">:</td><td class="info">D:\temp\Autopilot_ec\Test\Test_autopilot_demo_ec\Autopilot_Mode_Logic\CTCCov\Data_sil_ec\profile.txt</td></tr>
<tr><td class="info">HTML generated at</td><td class="info">:</td><td class="info">Thu May 21 05:44:18 2020</td></tr>
<tr><td class="info">Structural threshold</td><td class="info">:</td><td class="infob">100 %</td></tr>
<tr><td class="info">Statement threshold</td><td class="info">:</td><td class="infob">100 %</td></tr>
</table><br>
<table>
<thead>
<tr><th>TER %</th><th>-</th><th colspan="2">decision</th><th>TER %</th><th>-</th><th colspan="2">statement</th><th>File</th></tr>
</thead>
<tbody>
<tr><td class="dirb" colspan="9"><a name="a1"></a>Directory: C:\temp\Autopilot_ec\Test\Test_autopilot_demo_ec\Autopilot_Mode_Logic\Autopilot_Mode_Logic_sil_sil_ec_ert_rtw</td></tr>
<tr><td class="below">98 %</td><td class="below">-</td><td class="below">(41/42)</td><td width="115"><img border="1" src="red.gif" width="98" height="5" alt=""><img border="1" src="white.gif" width="2" height="5" alt=""></td><td class="below">99 %</td><td class="below">-</td><td class="below">(86/87)</td><td width="115"><img border="1" src="red.gif" width="99" height="5" alt=""><img border="1" src="white.gif" width="1" height="5" alt=""></td><td><a href="indexD1.html" class="underline">Autopilot_Mode_Logic_sil_sil_ec.c</a></td></tr>
<tr><td class="above">100 %</td><td class="above"></td><td class="above">(0/0)</td><td width="115"><img border="1" src="blue.gif" width="100" height="5" alt=""></td><td class="above">100 %</td><td class="above"></td><td class="above">(0/0)</td><td width="115"><img border="1" src="blue.gif" width="100" height="5" alt=""></td><td><a href="indexD2.html" class="underline">Autopilot_Mode_Logic_sil_sil_ec_data.c</a></td></tr>
<tr><td class="belowb">98 %</td><td class="below">-</td><td class="below">(41/42)</td><td width="115"><img border="1" src="red.gif" width="98" height="5" alt=""><img border="1" src="white.gif" width="2" height="5" alt=""></td><td class="belowb">99 %</td><td class="below">-</td><td class="below">(86/87)</td><td width="115"><img border="1" src="red.gif" width="99" height="5" alt=""><img border="1" src="white.gif" width="1" height="5" alt=""></td><td class="dirb"><a title="C:\temp\R2018B\Autopilot_ec\Test\Test_autopilot_demo_ec\Autopilot_Mode_Logic\Autopilot_Mode_Logic_sil_sil_ec_ert_rtw">DIRECTORY OVERALL</a></td></tr>
<tr><td class="ruler" colspan="9"> </td></tr>
<tr><td class="belowb">98 %</td><td class="below">-</td><td class="below">(41/42)</td><td width="115"><img border="1" src="red.gif" width="98" height="5" alt=""><img border="1" src="white.gif" width="2" height="5" alt=""></td><td class="belowb">99 %</td><td class="below">-</td><td class="below">(86/87)</td><td width="115"><img border="1" src="red.gif" width="99" height="5" alt=""><img border="1" src="white.gif" width="1" height="5" alt=""></td><td><a href="indexO.html" class="underlineb">OVERALL</a></td></tr>
</tbody>
</table><br>
</body>
</html>
MQC extracts the following information, stores and transforms it to the MQC data structure, so that the imported data can be added to the ValueFact table as rows:
from the first
<table>
header:read the last
<td>
of the<tr>
with the first<td>
equal to ‘Listing produced at’ (stored in MQC asReportDateTime
)
from the second
<table>
header:from
<thead>
start from third<th>
and every three<th>
read asMeasureName
.from each
<tr>
:read last
<td>
as<ArtifactName>
from
<td>
correspondent to<th>
read values for that measure in such a way that the first number is fetch asReached
and second asTotal
variable. (e.g for second row in sample report above, (41/42) is read as decision.Reached = 41 and decision.Total = 42)
Note
In reading rows from second table we will ignore
<tr>
with just one<td>
or has class=”dirb” or class=”ruler” or when last<td>
is equal to “DIRECTORY OVERALL” or “OVERALL”.
8.9. Danlawinc MxSuite¶
MQC reads data from the XML file named Report.RegResults.xml:
<MxVDevReportFile>
<Report>
<Date>20200423</Date>
<Time>15:39</Time>
<ProgramVersion>3.41.1.45984</ProgramVersion>
</Report>
<Regression>
<Overview>
<Project>
<Name>CRRCmil</Name>
<Description>
</Description>
<Folder>D:\CITest\CRRCmil</Folder>
<FileName>CRRCmil.mxp</FileName>
<ScenarioFolder>.\Regression\Regression-Set1-2020-04-23-15-39-11\ScenariosAndTestCases\</ScenarioFolder>
</Project>
</Overview>
<StatisticsTotals Total="1" PercentTotal="100" Passed="0" PercentPassed="0" Failed="1" PercentFailed="100" Skipped="0" PercentSkipped="0" RunTimeError="0" PercentRunTimeError="0" Missing="0" PercentMissing="0">
<Testcases Total="1" PercentTotal="100" Passed="0" PercentPassed="0" Failed="1" PercentFailed="100" Skipped="0" PercentSkipped="0" RunTimeError="0" PercentRunTimeError="0" Missing="0" PercentMissing="0" />
</StatisticsTotals>
</Regression>
</MxVDevReportFile>
MQC reads out
from the
<Report>
<Date>
and<Time>
(stored in MQC asReportDateTime
)
from each
<Regression>
header:<Overview><Project><Name>
(stored in MQC asArtifactName
)from
<StatisticsTotals>
header:Total
(stored in MQC asScenarios.Total
)Passed
(stored in MQC asScenarios.Passed
)Failed
(stored in MQC asScenarios.Failed
)Skipped
(stored in MQC asScenarios.Skipped
)RunTimeError
(stored in MQC asScenarios.RunTimeError
)Missing
(stored in MQC asScenarios.Missing
)
from
<Testcases>
header:Total
(stored in MQC asTestcases.Total
)Passed
(stored in MQC asTestcases.Passed
)Failed
(stored in MQC asTestcases.Failed
)Skipped
(stored in MQC asTestcases.Skipped
)RunTimeError
(stored in MQC asTestcases.RunTimeError
)Missing
(stored in MQC asTestcases.Missing
)
8.10. SL Check Model Advisor¶
MQC reads from the HTML file:
<div class="ReportContent" id="Artifact">
<table class="AdvTableNoBorder" width="100%" border="0">
<tr>
<td align="left" valign="top"><b>System: <font color="#800000">Artifact</font></b></td>
<td align="right" valign="top"><b>Current run: <font color="#800000">22-Oct-2020 18:13:17</font></b></td>
</tr>
</table>
<b>Run Summary</b></font><br/>
<table class="AdvTableNoBorder" width="60%" border="0">
<tr>
<th align="left" valign="top"><b>Pass</b></th>
<th align="left" valign="top"><b>Fail</b></th>
<th align="left" valign="top"><b>Warning</b></th>
<th align="left" valign="top"><b>Not Run</b></th>
<th align="left" valign="top"><b>Total</b></th>
</tr>
<tr>
<td align="left" valign="top" >  <img src="task_passed.png" /> 38</td>
<td align="left" valign="top" >  <img src="task_failed.png" /> 8</td>
<td align="left" valign="top" >  <img src="task_warning.png" /> 7</td>
<td align="left" valign="top" >  <img src="icon_task.png" /> 363</td>
<td align="left" valign="top" >416</td>
</tr>
</table>
</div>
MQC extracts the following information, stores and transforms it to the MQC data structure:
the
ArtifactName
fromSystem:
the
ReportDateTime
fromCurrent Run:
the complete
Run Summary
from the second table inside the<div class="ReportContent">
tag:Pass
asGuidelineCount.Passed
Fail
asGuidelineCount.Failed
Warning
asGuidelineCount.Warning
Not Run
asGuidelineCount.Not Run
Total
asGuidelineCount.Total
8.11. Simulink Design Verifier¶
<div class="titlepage">
<div>
<div><h1 class="title"><a name="d0e1"></a>Simulink Design Verifier Report</h1></div>
<div><h2 class="subtitle">Artifact</h2></div>
<div><div class="author"><h3 class="author"><span class="firstname">Author</span></h3></div></div>
<div><p class="pubdate">27-Oct-2020 12:03:54</p></div>
</div>
</div>
<div class="chapter" title="Chapter 1. Summary">
<div class="titlepage">
<div><h2 class="title"><a name="d0e13"></a>Chapter 1. Summary</h2></div>
</div>
<p title="Analysis Information"><b>Analysis Information </b><a name="d0e22"></a></p>
<div class="table">
<div class="table-contents">
<table summary="" border="0" cellspacing="0" fastRender="1">
<tbody>
<tr><td align="left">Model:</td><td align="left">Artifact</td></tr>
<tr><td align="left">Mode:</td><td align="left">Design error detection</td></tr>
<tr><td align="left">Status:</td><td align="left">Completed normally</td></tr>
<tr><td align="left">Analysis Time:</td><td align="left">66s</td></tr>
</tbody>
</table>
</div>
</div>
<p title="Objectives Status"><b>Objectives Status </b><a name="d0e51"></a></p>
<div class="table">
<div class="table-contents">
<table summary="" border="0" cellspacing="0" fastRender="1">
<thead>
<tr><th align="left">Number of Objectives:</th><th align="left">5</th></tr>
</thead>
<tbody>
<tr><td align="left">Objectives Proven Valid: </td><td align="left">4</td></tr>
<tr><td align="left">Objectives Falsified with Test Cases: </td><td align="left">1</td></tr>
</tbody>
</table>
</div>
</div>
</div>
MQC extracts the following information, stores and transforms it to the MQC data structure:
the
ArtifactName
from the<h2 class="subtitle">
tagthe
ReportDateTime
from the<p class="pubdate">
tagthe
MeasurementName
from theAnalysis Information
section of the “Summary” chapterfrom the
Objectives Status
section of the “Summary” chapter:Number of Objectives
asObjectives.Total
Objectives Proven Valid
asObjectives.Passed
Objectives Falsified with Test Cases
asObjectives.Failed
8.12. QA-System QAC¶
MQC supports two types of QA-System QAC report formats:
8.12.1. Standard XML report format¶
MQC will read from the XML file:
<AnalysisData timestamp="20200423T162554" projectpath="C:/Users/public/AppData/Local/samples/Examples" reportpath="C:/Users/public/AppData/Local/samples/Examples/configs/reports">
<dataroot type="project">
<tree type = "rules" >
<RuleGroup name="xxx" total="5212" active="1112" >
<Rule id = "" total="5" active="2" text="" >
<Rule id = "" total="2" active="2" text="" >
<Message guid = "" total="2" active="2" text="" />
</Rule>
</Rule>
<Rule id = "" total="3" active="0" text="" >
<Rule id = "" total="3" active="0" text="" >
<Message guid = "" total="1" active="0" text="" />
<Message guid = "" total="2" active="0" text="" />
</Rule>
</Rule>
</RuleGroup>
</tree>
</dataroot>
<dataroot type="per-file">
<File path=".../example.h">
<tree type="rules">
<RuleGroup name = "xxx" total="84" active="53" >
<Rule id="xxx" total="29" active="23" text="There shall be no occurrence of undefined or critical unspecified behaviour" >
<Message guid = "qac-9.3.1-0602" total="6" active="0" text="" />
<Message guid = "qac-9.3.1-0603" total="12" active="12" text="" />
<Message guid = "qac-9.3.1-0836" total="1" active="1" text="" />
<Message guid = "qac-9.3.1-0848" total="5" active="5" text="" />
<Message guid = "qac-9.3.1-0854" total="5" active="5" text="" />
</Rule>
</RuleGroup>
</tree>
<tree type="levels">
<Level guid = "QA_WARNING" total="55" active="30" name="Warnings" ></Level>
<Level guid = "QA_ERROR" total="2" active="2" name="Errors" ></Level>
</tree>
</File>
</dataroot>
</AnalysisData>
MQC extracts the following information, stores and transforms it to the MQC data structure, so that the imported data can be added to the ValueFact table as rows:
from the
<AnalysisData>
header:timestamp
(stored in MQC asReportDateTime
)
from each
<File>
header inside<dataroot type="per-file">
:path
(stored in MQC asArtifactPath
)from the first
<RuleGroup>
inside<tree type="rules">
:total
(stored in MQC asViolation Count.Total
)active
(stored in MQC asViolation Count.Active
)
for each
<Level>
inside<tree type="levels">
:read
active
if guid = “QA_WARNING” and stored in MQC asActive Diagnostics.WARNING
read
active
if guid = “QA_ERROR” and stored in MQC asActive Diagnostics.ERROR
read sum of both above and stored in MQC as
Active Diagnostics.Active
From
<tree type="rules">
header:read number of
<Rule>
headers that contained just<Message>
as child element with active=”0” and stored in MQC asViolated Rules.Violated
read number of
<Rule>
headers that contained just<Message>
as child element and stored in MQC asViolated Rules.Total
From
<tree type="rules">
inside<dataroot type="per-file">
header:read number of
<Rule>
headers that contained just<Message>
as child element with active=”0” and stored in MQC asProject Rules.Violated
read number of
<Rule>
headers that contained just<Message>
as child element and stored in MQC asProject Rules.Total
8.12.2. Standard HTML report format¶
MQC will read from the HTML file:
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<div id = "head" >
<div class="stitle">Project : C:/Users/public/AppData/Local/Examples<br/> Status at: 23 Apr, 2019 at 16:25:13</div>
</div>
<div class="head">
<div class="toc">
<ol>
<li><a href="#DiagsPerParents">Diagnostics Per Parent Rules</a></li>
<li><a href="#WorstRules">Most Violated Rules</a></li>
<li><a href="#FileStatus">File Status</a></li>
<li><a href="#AnalysisSettings">Analysis Settings</a></li>
<li><a href="#StatCalc">Calculation Information</a></li>
<br/> <br/> <br/> </ol>
</div>
<div class="summary">
<table border="0">
<col border="50"/>
<tr><td>Number of Files</td><td>185</td></tr>
<tr><td>Lines of Code (source files only)</td><td>16431</td></tr>
<tr><td>Total preprocessed code line</td><td>3036</td></tr>
<tr><td>Diagnostic Count</td><td>948</td></tr>
<tr><td>Rule Violation Count</td><td>1212</td></tr>
<tr><td>Violated Rules</td><td>183</td></tr>
<tr><td>Compliant Rules</td><td>7</td></tr>
<tr><td>File Compliance Index</td><td>97.41%</td></tr>
<tr><td>Project Compliance Index</td><td>6.68%</td></tr>
</table>
</div>
</div>
<div id="content">
<div class="analstat">
<table border="1" id="filestat" class="tablesorter">
<thead>
<tr><th>Files</th><th>Active Diagnostics</th><th>Violated Rules</th><th>Violation Count</th><th>Compliance Index</th></tr>
</thead>
<tbody>
<tr><td align="left"><a href="..\src\Example.c" title="..\src\Example.c">Example.c</a></td><td align="right">4</td><td align="right">1</td><td align="right">4</td><td align="right">99.42%</td></tr>
</tbody>
</table>
</div>
</div>
</body>
</html>
MQC extracts the following information, stores and transforms it to the MQC data structure, so that the imported data can be added to the ValueFact table as rows:
from
<div class="stitle">
that contains ‘Status at:’ read date and time and stored in MQC asReportDateTime
from
<table id="filestat">
header:from
<thead>
read<th>
asMeasureName
(just ignore first and last one).from each
<tr>
:read first
<td>
as<ArtifactName>
from
<td>
correspondent to<th>
read values for that measure.
from
<table>
inside<div class="summary">
:read second
<td>
of<tr>
that first is equal to ‘Violated Rules’ (stored in MQC asProject Rules.Violated
)read second
<td>
of<tr>
that first is equal to ‘Compliant Rules’ (stored in MQC asProject Rules.Compliant
)
8.13. Manual data import for unsupported data sources (Excel Template)¶
MQC provides the possibility to import data from any other data source using the manual import option.
Note
All measures imported into MQC have to be configured in the Quality Model (see Base Measures and Default Values). Imported but not configured measures will be ignored in all visualizations as well as for the quality calculation.
The manual import has to be done using Excel. From the
Export Template (Manual
Import)
button to create an Excel import file that can be used to load data
into MQC.
Select one of the following options to customize the data import template according to your needs:
All
: The Excel file already contains a column for each measure configured in the quality model as well as one row for each artifact imported respectively configured in the project structure.Filtered
: The Excel file only contains those measures and artifacts selected via the filter panel on the right-hand side of the pages.Marked
: The Excel file only contains those measures and artifacts that were marked by the user, e.g. if the user have selected an artifact KPI and a specific data source KPI at the Data Status page, the file only contains a row for the marked artifact as well as only columns for measures belonging to the marked data source.
The column Artifact
consists of entries representing the objects for which
data shall be collected and for which quality shall be computed (e.g. Simulink
models, requirements documents, software components).
The entries of the column ReportDateTime
are considered by MQC as the days
when raising the data, which shall be collected. Those configured report dates
are used to assign the imported data to MQC revisions.
Finally, all other columns are representing measures for which data shall be
imported into MQC. As shown in Figure 8.2, measures (i.e. the
column names) have to follow the syntax
DataSource.Measurement.BaseMeasure.Variable
.
DataSource
: Specify where your data is coming from (e.g. the name of the tool which produced the data).Measurement
: Provide more structuring regarding the data (e.g. the reason why data is collected, the test environment etc.). This is optional and may be left empty. In this case the measure name syntax isDataSource.BaseMeasure.Variable
. Nevertheless, it is recommended to fill outMeasurement
to achieve a high compliance to ISO 250xx.BaseMeasure
: Specifies a group of measures.Variable
: The name of the specific measure belonging to the base measure group.
The value for a specific measure for an artifact at a certain report date then has to be assigned to the corresponding cell within the Excel template.
In case you might want to assign default values for your imported base measures, you have to define them in compliance with this syntax within the Quality Model Base Measure excel sheet (see Figure 12.1).
Note
The manual data import supports .xls files, in addition to .xlsx files, and multiple sheets starting from MQC 4.3.