8. Data Sources¶
To import data from reports in MQC, the data has to be provided as files. These files can be located in the local file system, a network path or a git repository.
A new data source can be added through a browser in the data source dialog. The type of the data source has to be selected first and then the respective directory, files or git repository can be added. You can also have multiple data sources added to your project.
Each data source is constantly monitored for updates and changes. Whenever e.g. a new report appears in a data source (new file in directory or new commit in version control system), it will be detected by MQC. Either the user is notified about the changes or the project is automatically updated (refer to Refresh Data).
Note
Use the Force Refresh
button (see
Figure 8.1) to not just apply changes to
a project but to re-import all data anew. This may be useful for instance
in case of the commit history of a version control system has been changed.
8.1. Local File System / Network / Sample Projects¶
The local file system and network types function similarly. A directory, and therefore all subdirectories and files within, or a single file can be selected as the data source. At any point, the allowed file extensions that can be read by MQC are displayed below.
Navigating through the file system works like a windows explorer. Double-clicking a directory opens it. The current path is displayed at the top of the browser and can be used to navigate upwards in the file system. You can also add the path manually.
Note
The local file system of the server is not accessible when using the web player.
You have also direct access to the MQC Showcase source files under Sample Projects
.
8.2. Version Control Systems¶
MQC also allows version control systems to be used as a data source. Currently only Git is supported.
8.2.1. Git (Beta)¶
After selecting Git as the data source type, all previously configured git repositories in the project are available for selection.
By clicking on “Manage repositories” git repositories can be added and already configured repositories can be deleted or modified.
Note
If git is not installed, locally (if using the MQC desktop client) or on the server (if using the web player), an error is shown.
Git for windows can be found here: https://git-scm.com/downloads
Git repositories can be added by ssh or https urls.
An rsa private key can be provided for authentication of ssh urls if it is not configured as the default ssh key for the current user.
For an https url the authentication has to be done by entering the username and password, if it is not already stored in the windows credential store.
When a repository is saved, it is validated. If the authentication fails, an error message is shown.
In case you have multiple repositories with a similar structure, you can configure them as one data source by adding a list of repository urls to the “Add new repositories” dialog and use one configuration (branches, commits, time range, …) for all repositories afterwards.
You can select a branch of a configured git repository(ies). For further configurations of the data source, the selected branch of the repository(ies) should be checked out first.
Note
Only branches that exist in all repositories can be selected, when multiple Git Repositories are given.
The selected branch for a git repository(ies) can be manually checked out using
the Fetch Repositories
button.
The selected branch can then be added as a data source so that it’s entire commit history can be used by MQC.
In case that multiple repositories are added, they are considered as one data source.
Since your commit history might contain commits not relevant for your MQC quality analysis, you have the possibility of filtering these commits in the Commit Filters section of this dialog.
The Commit Filters section provides three possible targets to filter, namely,
Message
, Author
and Tag
. You can further choose to either include
or exclude the commits matching this filter by clicking on the toggle button
below Apply
. The green colored +
stands for include and the red colored
-
stands for exclude. Next you need to add the suitable regular expression
in RegEx
text box and the filtered commits matching all the configured
filters can be seen on the Filtered Commits
panel at the right side. All
the filters to the same target are connected with an OR
whereas different
targets are connected with an AND
.
For example, the configured filters as seen in the image below would be
interpreted as, All commits from the Author "Jenkins Slave" AND with the
Message "Static Analysis"
.
If there is another filter for Message
then it would be,
All commits from the Author "Jenkins Slave" AND with (Message "Static
Analysis" OR Message "Dynamic Analysis")
.
Additionally, MQC offers the possibility to filter by time. This allows to consider only commits of a specific time frame.
Note
If no time filter is added, but a project structure configuration has been loaded, MQC automatically sets the project start and end dates as internal time restriction commit filter to limit the number of commits to be recognized especially for very large repositories.
In that case and if a commit appears outside the configured project time frame, you will be notified about such commits within the Notifications panel. After adapting the project structure configuration, a data refresh is necessary to fetch the relevant commits from the repository.
Once you have filtered the relevant commits, you might also want to select specific folders or files relevant to your analysis. MQC provides two possibilities to do this.
If you select the Regular Expressions
option in
the drop down, then you can add multiple regular expressions as done in the
Commit Filters section.
If you select Browser Selection
option, one file or folder can be chosen
as the data source, similar to the selection for local or network.
Note
Browser Selection
is available only when there is one
git repository given. By multiple repositories only Regular Expressions
could be applied.
Finally, with the checkbox shown in Figure 8.16 you can define which date and time is used as the report date-time:
Git commit date (checked)
Report creation date time as contained in the report itself (unchecked)
If a git data source has been added, a monitoring service periodically checks if a new commit is made on the remote. The default for the check interval is set to 10 seconds. In case of multiple repositories, the supervision timer is stepwise increased for performance reasons based on the number of repositories. I.e. If you have configured more than 100 repositories, the check happens only every 60 seconds.
When data is refreshed, the git repository is updated and all remote changes are fetched and applied to the MQC project.