[edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC
Guo, Gua
gua.guo at intel.com
Wed Sep 14 02:11:02 UTC 2022
@Sean Brogan<mailto:spbrogan at outlook.com>
I go through the code Merged PR 5008: Initial implementation or code coverage on basecore u… · microsoft/mu_basecore at f6af51f (github.com)<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>
It looks good from my view point.
@Kinney, Michael D<mailto:michael.d.kinney at intel.com>
Does that pytool plugin support viewing the code coverage visually in C source files? Yes
Either HTML or preferably an Visual Studio Code plugin? I think it can be supported by both based on below change. Due to VScode plugin is auto scan *.gcda and show the result in VScode source tree.
Merged PR 5008: Initial implementation or code coverage on basecore u… · microsoft/mu_basecore at f6af51f (github.com)<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>
Flow for stuart_ci_build: (a) -> (b) -> (c) -> (d) -> (e) -> (f)
Flow for vscode extension Gcov Viewer - Visual Studio Marketplace<https://marketplace.visualstudio.com/items?itemName=JacquesLucke.gcov-viewer>: (a) -> (b) -> (c) -> (d) -> (g)
(a) stuart_ci_build -p XXXPkg
(b) Call Edk2/.pytool/Plugin/HostUnitTestCompilerPlugin/HostUnitTestCompilerPlugin.py
(c) Call Edk2/BaseTools/Plugin/HostBasedUnitTestRunner/HostBasedUnitTestRunner.py
(d) execute Build/XXXPkg/**/*Test* -> generate *.gcda files
(e) use “lcov” tool to parse *.gcda
(f) use “pycobertura” tool to generate index.html
(g) vscode extension to parse *.gcda and show it on source code directly.
Can it be updated to support VS compilers tool? No, currently I don’t find a good way to generate code coverage for MSVC compiler
Thanks,
Gua
From: Kinney, Michael D <michael.d.kinney at intel.com>
Sent: Wednesday, September 14, 2022 9:52 AM
To: Sean Brogan <spbrogan at outlook.com>; devel at edk2.groups.io; Guo, Gua <gua.guo at intel.com>; Kinney, Michael D <michael.d.kinney at intel.com>
Cc: Sean Brogan <sean.brogan at microsoft.com>; Michael Kubacki <mikuback at linux.microsoft.com>
Subject: RE: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC
Does that pytool plugin support viewing the code coverage visually in C source files?
Either HTML or preferably an Visual Studio Code plugin?
Can it be updated to support VS compilers tool?
Mie
From: Sean Brogan <spbrogan at outlook.com<mailto:spbrogan at outlook.com>>
Sent: Tuesday, September 13, 2022 6:18 PM
To: devel at edk2.groups.io<mailto:devel at edk2.groups.io>; Guo, Gua <gua.guo at intel.com<mailto:gua.guo at intel.com>>
Cc: Kinney, Michael D <michael.d.kinney at intel.com<mailto:michael.d.kinney at intel.com>>; Sean Brogan <sean.brogan at microsoft.com<mailto:sean.brogan at microsoft.com>>; Michael Kubacki <mikuback at linux.microsoft.com<mailto:mikuback at linux.microsoft.com>>
Subject: Re: [edk2-devel] [PATCH v2] UnitTestFrameworkPkg: Add code coverage support for GCC
Hi Gua Guo,
I would propose that we make this easier for both the local user and the CI system to get coverage metrics.
Can you review this commit as it uses the same Edk2 plugins but adds code coverage support with GCC5:
Merged PR 5008: Initial implementation or code coverage on basecore u… · microsoft/mu_basecore at f6af51f (github.com)<https://github.com/microsoft/mu_basecore/commit/f6af51f856747faec5a9f78f5604a69211b5af2d#diff-52e599c27ab877ef616f244e8b26826a53853872acb894df4aeff99f48df373d>
Thanks
Sean
On 9/13/2022 6:01 PM, Guo, Gua wrote:
From: Gua Guo <gua.guo at intel.com><mailto:gua.guo at intel.com>
In order to collect code coverage after running executable
file, generate *.gcda and *.gcno file that require by lcov tool
to generate code coverage report.
Cc: Michael D Kinney <michael.d.kinney at intel.com><mailto:michael.d.kinney at intel.com>
Cc: Sean Brogan <sean.brogan at microsoft.com><mailto:sean.brogan at microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew at microsoft.com><mailto:Bret.Barkelew at microsoft.com>
Cc: Michael Kubacki <mikuback at linux.microsoft.com><mailto:mikuback at linux.microsoft.com>
Signed-off-by: Gua Guo <gua.guo at intel.com><mailto:gua.guo at intel.com>
---
UnitTestFrameworkPkg/ReadMe.md | 116 +++++++++++++++++++-
UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc | 2 +-
2 files changed, 116 insertions(+), 2 deletions(-)
diff --git a/UnitTestFrameworkPkg/ReadMe.md b/UnitTestFrameworkPkg/ReadMe.md
index e696412cb3..93fdacfc4a 100644
--- a/UnitTestFrameworkPkg/ReadMe.md
+++ b/UnitTestFrameworkPkg/ReadMe.md
@@ -361,7 +361,7 @@ RUNNING TEST SUITE: Int Safe Conversions Test Suite
```
You can also, if you are so inclined, read the output from the exact instance of the test that was run during
-`stuart_ci_build`. The ouput file can be found on a path that looks like:
+`stuart_ci_build`. The output file can be found on a path that looks like:
`Build/<Package>/HostTest/<Arch>/<TestName>.<TestSuiteName>.<Arch>.result.xml`
@@ -399,6 +399,120 @@ CMOCKA_XML_FILE=<absolute or relative path to output file>
This mode is used by the test running plugin to aggregate the results for CI test status reporting in the web view.
+### Enable Code Coverage for GCC host-based tests
+
+LCOV is an extension of GCOV, a GNU tool which provides information about what parts of a program are actually executed (i.e. "covered") while running a particular test case. The extension consists of a set of scripts which build on the textual GCOV output to implement it.
+
+Documentation for LCOV can be found here:
+https://lcov.readthedocs.io/
+
+You can follow below steps to generate your code coverage report.
+
+
+
+```bash
+user at linux_machine:~/_uefi$stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -p MdePkg -t NOOPT
+...
+user at linux_machine:~/_uefi$lcov -c --rc lcov_branch_coverage=1 -o UnitTest.info -d Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/
+Capturing coverage data from Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/
+Found gcov version: 9.4.0
+Using intermediate gcov format
+Scanning Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/ for .gcda files ...
+Found 38 data files in Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/RShiftU64.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Math64.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SafeString.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes32.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CpuDeadLoop.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x64.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LongJump.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SetJump.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/BitField.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/String.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32Remainder.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/X64/GccInline.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultU64x32.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/MultS64x64.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LinkedList.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/LShiftU64.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/DivU64x32.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/SwapBytes16.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/Unaligned.gcda
+Processing Library/BaseLib/UnitTestHostBaseLib/OUTPUT/CheckSum.gcda
+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMem.gcda
+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGuid.gcda
+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CompareMemWrapper.gcda
+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMemWrapper.gcda
+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem64Wrapper.gcda
+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/MemLibGeneric.gcda
+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/CopyMemWrapper.gcda
+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/SetMem.gcda
+Processing Library/BaseMemoryLib/BaseMemoryLib/OUTPUT/ZeroMemWrapper.gcda
+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLibInternal.gcda
+Processing Library/BasePrintLib/BasePrintLib/OUTPUT/PrintLib.gcda
+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib64.gcda
+Processing Library/BaseSafeIntLib/BaseSafeIntLib/OUTPUT/SafeIntLib.gcda
+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/Base64UnitTest.gcda
+Processing Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/OUTPUT/AutoGen.gcda
+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/SafeIntLibUintnIntnUnitTests64.gcda
+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/TestBaseSafeIntLib.gcda
+Processing Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/OUTPUT/AutoGen.gcda
+
+user at linux_machine:~/_uefi$genhtml UnitTest.info --branch-coverage -o Build/UnitTestReport
+
+Reading data file UnitTest.info
+Found 38 entries.
+Found common filename prefix "~/_uefi/MdePkg/Library"
+Writing .css and .png files.
+Generating output.
+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseLib/BaseLibUnitTestsHost/DEBUG/AutoGen.c
+Processing file ~/_uefi/Build/MdePkg/HostTest/NOOPT_GCC5/X64/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLibHost/DEBUG/AutoGen.c
+Processing file BaseLib/Unaligned.c
+Processing file BaseLib/CheckSum.c
+Processing file BaseLib/BitField.c
+Processing file BaseLib/DivU64x32.c
+Processing file BaseLib/CpuDeadLoop.c
+Processing file BaseLib/LongJump.c
+Processing file BaseLib/MultU64x64.c
+Processing file BaseLib/RShiftU64.c
+Processing file BaseLib/SwapBytes32.c
+Processing file BaseLib/SwapBytes16.c
+Processing file BaseLib/SetJump.c
+Processing file BaseLib/LinkedList.c
+Processing file BaseLib/LShiftU64.c
+Processing file BaseLib/SafeString.c
+Processing file BaseLib/String.c
+Processing file BaseLib/DivU64x32Remainder.c
+Processing file BaseLib/MultS64x64.c
+Processing file BaseLib/MultU64x32.c
+Processing file BaseLib/Math64.c
+Processing file BaseLib/X64/GccInline.c
+Processing file BaseMemoryLib/SetMemWrapper.c
+Processing file BaseMemoryLib/CompareMemWrapper.c
+Processing file BaseMemoryLib/SetMem64Wrapper.c
+Processing file BaseMemoryLib/ZeroMemWrapper.c
+Processing file BaseMemoryLib/CopyMemWrapper.c
+Processing file BaseMemoryLib/CopyMem.c
+Processing file BaseMemoryLib/SetMem.c
+Processing file BaseMemoryLib/MemLibGuid.c
+Processing file BaseMemoryLib/MemLibGeneric.c
+Processing file BasePrintLib/PrintLib.c
+Processing file BasePrintLib/PrintLibInternal.c
+Processing file BaseSafeIntLib/SafeIntLib64.c
+Processing file BaseSafeIntLib/SafeIntLib.c
+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseLib/Base64UnitTest.c
+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/TestBaseSafeIntLib.c
+Processing file ~/_uefi/MdePkg/Test/UnitTest/Library/BaseSafeIntLib/SafeIntLibUintnIntnUnitTests64.c
+Writing directory view page.
+Overall coverage rate:
+ lines......: 58.6% (2688 of 4589 lines)
+ functions..: 57.1% (249 of 436 functions)
+ branches...: 25.3% (1062 of 4195 branches)
+
+...
+
+```
+
### Important Note
This works on both Windows and Linux, but is currently limited to x64 architectures. Working on getting others, but we
diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
index 4dd8d4ac67..8623aecb33 100644
--- a/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
+++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkgHost.dsc.inc
@@ -19,7 +19,7 @@
MemoryAllocationLib|UnitTestFrameworkPkg/Library/Posix/MemoryAllocationLibPosix/MemoryAllocationLibPosix.inf
[BuildOptions]
- GCC:*_*_*_CC_FLAGS = -fno-pie
+ GCC:*_*_*_CC_FLAGS = -fno-pie -ftest-coverage -fprofile-arcs
!ifdef $(UNIT_TESTING_DEBUG)
MSFT:*_*_*_CC_FLAGS = -D UNIT_TESTING_DEBUG=1
GCC:*_*_*_CC_FLAGS = -D UNIT_TESTING_DEBUG=1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#93735): https://edk2.groups.io/g/devel/message/93735
Mute This Topic: https://groups.io/mt/93669135/1813853
Group Owner: devel+owner at edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [edk2-devel-archive at redhat.com]
-=-=-=-=-=-=-=-=-=-=-=-
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/edk2-devel-archive/attachments/20220914/d165b717/attachment-0001.htm>
More information about the edk2-devel-archive
mailing list