[edk2-devel] running CI locally
Sean
spbrogan at outlook.com
Fri Aug 21 21:50:47 UTC 2020
sorry. In my last email I started typing at top and then moved inline
and didn't delete my initial comments. Please ignore the comments on
top and look inline for details. :)
Thanks
Sean
On 8/21/2020 2:36 PM, Sean wrote:
> Laszlo,
>
>
>
> Idea being if the package supports host based unit tests it will scan
> package to make sure all host based unit tests and libraries are listed
> in the DSC.
>
> As to why you see a different set of packages...that is interesting. If
> you can include your full build logs i can look a little closer. I know
> on the Azure CI servers we don't run the
>
>
> On 8/21/2020 12:23 AM, Laszlo Ersek wrote:
>> Hi Mike, Sean;
>>
>> On 08/19/20 19:59, Laszlo Ersek wrote:
>>
>>> I'll report back with more results.
>>
>> I installed a Fedora 32 Server VM.
>>
>> Installed the "mono-complete" package (+its deps).
>>
>> Installed the "nodejs" package.
>>
>> Installed "cspell" (+deps) with "npm" (with the latter coming from the
>> nodejs package).
>>
>> Edk2 checked out at current HEAD
>> (5a6d764e1d073d28e8f398289ccb5592bf9a72ba).
>>
>>
>> * Platform CI results:
>>
>> - <https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI>:
>>
>> Built successfully for IA32, X64, and IA32X64 (toolchain: GCC5).
>>
>> - <https://github.com/tianocore/edk2/tree/master/ArmVirtPkg/PlatformCI>:
>>
>> Built successfully for ARM and AARCH64 (toolchain: GCC5).
>>
>>
>> * Core CI results:
>>
>> - <https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md>
>>
>> Complete run successful (toolchain: GCC5); no package / arch / target
>> / test restrictions.
>>
>> I got the following WARNINGs:
>>
>> - ArmVirtPkg, EmulatorPkg, and OvmfPkg are covered by Platform CI, not
>> Core CI -- these are expected:
>>
>>> PROGRESS - --Running ArmVirtPkg: Compiler Plugin DEBUG --
>>> WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG
>>> PROGRESS - --Running ArmVirtPkg: Compiler Plugin RELEASE --
>>> WARNING - --->Test Skipped: in plugin! Compiler Plugin RELEASE
>>
>>> PROGRESS - --Running EmulatorPkg: Compiler Plugin DEBUG --
>>> WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG
>>> PROGRESS - --Running EmulatorPkg: Compiler Plugin RELEASE --
>>> WARNING - --->Test Skipped: in plugin! Compiler Plugin RELEASE
>>
>>> PROGRESS - --Running OvmfPkg: Compiler Plugin DEBUG --
>>> WARNING - --->Test Skipped: in plugin! Compiler Plugin DEBUG
>>> PROGRESS - --Running OvmfPkg: Compiler Plugin RELEASE --
>>> WARNING - --->Test Skipped: in plugin! Compiler Plugin RELEASE
>>
>> - ArmVirtPkg, DynamicTablesPkg, CryptoPkg, EmulatorPkg, FatPkg,
>> NetworkPkg, OvmfPkg, PcAtChipsetPkg, SecurityPkg and ShellPkg are
>> not covered by host-based unit tests -- I think also expected:
>>
>>> PROGRESS - --Running ArmVirtPkg: Host Unit Test Compiler Plugin NOOPT --
>>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin
>>> NOOPT
>>
>>> PROGRESS - --Running DynamicTablesPkg: Host Unit Test Compiler Plugin
>>> NOOPT --
>>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin
>>> NOOPT
>>
>>> PROGRESS - --Running CryptoPkg: Host Unit Test Compiler Plugin NOOPT --
>>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin
>>> NOOPT
>>
>>> PROGRESS - --Running EmulatorPkg: Host Unit Test Compiler Plugin
>>> NOOPT --
>>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin
>>> NOOPT
>>
>>> PROGRESS - --Running FatPkg: Host Unit Test Compiler Plugin NOOPT --
>>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin
>>> NOOPT
>>
>>> PROGRESS - --Running NetworkPkg: Host Unit Test Compiler Plugin NOOPT --
>>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin
>>> NOOPT
>>
>>> PROGRESS - --Running OvmfPkg: Host Unit Test Compiler Plugin NOOPT --
>>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin
>>> NOOPT
>>
>>> PROGRESS - --Running PcAtChipsetPkg: Host Unit Test Compiler Plugin
>>> NOOPT --
>>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin
>>> NOOPT
>>
>>> PROGRESS - --Running SecurityPkg: Host Unit Test Compiler Plugin
>>> NOOPT --
>>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin
>>> NOOPT
>>
>>> PROGRESS - --Running ShellPkg: Host Unit Test Compiler Plugin NOOPT --
>>> WARNING - --->Test Skipped: in plugin! Host Unit Test Compiler Plugin
>>> NOOPT
>>
>> - I'm not really sure about the following warnings. They were emitted
>> for a subset of the above packages. I read the docs at
>>
>> <https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md#host-module-inclusion-test---hostunittestdsccompletecheck>
>>
>> but I still don't understand :)
>
> Each plugin/test has a readme as well. Not sure if this helps explain
> the HostUnitTestDscCompleteCheck more.
>
> https://github.com/tianocore/edk2/tree/master/.pytool/Plugin/HostUnitTestDscCompleteCheck
>
>
> As to why you don't see it skipped for ArmVirtPkg or a few others: I
> think this is a bug.
>
> In Azure CI i see it as test passed.
>
> PROGRESS - --Running ArmVirtPkg: Host Unit Test Dsc Complete Check Test
> NO-TARGET --
> PROGRESS - --->Test Success: Host Unit Test Dsc Complete Check Test
> NO-TARGET
>
> When i look at the code
> https://github.com/tianocore/edk2/blob/master/.pytool/Plugin/HostUnitTestDscCompleteCheck/HostUnitTestDscCompleteCheck.py#L59
>
>
> I see that how those packages *.ci.yaml have it configured is different
> than those that show skipped below. It avoids the skip conditions but
> since those packages don't have any host unit test libraries or modules
> there is no failure.
>
> This should probably be a bugzilla on the
> HostUnitTestDscCompleteCheck.py as the reporting is not correct. Test
> should show up as skipped.
>
>
> You can see them here showing up as success
> https://dev.azure.com/tianocore/edk2-ci/_build/results?buildId=11309&view=logs&j=780eaa17-fc2d-5131-b9d2-21b19bc9bec5&t=0791c90f-e7d8-5372-c9c5-edbe30bf6aeb&l=32
>
>
>>
>>> PROGRESS - --Running CryptoPkg: Host Unit Test Dsc Complete Check
>>> Test NO-TARGET --
>>> WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete
>>> Check Test NO-TARGET
>>
>>> PROGRESS - --Running FatPkg: Host Unit Test Dsc Complete Check Test
>>> NO-TARGET --
>>> WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete
>>> Check Test NO-TARGET
>>
>>> PROGRESS - --Running NetworkPkg: Host Unit Test Dsc Complete Check
>>> Test NO-TARGET --
>>> WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete
>>> Check Test NO-TARGET
>>
>>> PROGRESS - --Running PcAtChipsetPkg: Host Unit Test Dsc Complete
>>> Check Test NO-TARGET --
>>> WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete
>>> Check Test NO-TARGET
>>
>>> PROGRESS - --Running SecurityPkg: Host Unit Test Dsc Complete Check
>>> Test NO-TARGET --
>>> WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete
>>> Check Test NO-TARGET
>>
>>> PROGRESS - --Running ShellPkg: Host Unit Test Dsc Complete Check Test
>>> NO-TARGET --
>>> WARNING - --->Test Skipped: in plugin! Host Unit Test Dsc Complete
>>> Check Test NO-TARGET
>>
>> - Still related to host-based unit tests, I believe the following
>> warnings, for FmpDevicePkg, MdeModulePkg, MdePkg, UefiCpuPkg, and
>> UnitTestFrameworkPkg, report that running the tests on the build
>> host (understandably) prevents the tests from covering all edk2
>> architectures:
>
> Yes this message is because we override the arch for NOOPT to match that
> of the build system since we are going to run these executables in the
> host environment.
>
>>
>>> PROGRESS - --Running FmpDevicePkg: Host Unit Test Compiler Plugin
>>> NOOPT --
>>> WARNING - Allowing Override for key TARGET_ARCH
>>
>>> PROGRESS - --Running MdeModulePkg: Host Unit Test Compiler Plugin
>>> NOOPT --
>>> WARNING - Allowing Override for key TARGET_ARCH
>>
>>> PROGRESS - --Running MdePkg: Host Unit Test Compiler Plugin NOOPT --
>>> WARNING - Allowing Override for key TARGET_ARCH
>>
>>> PROGRESS - --Running UefiCpuPkg: Host Unit Test Compiler Plugin NOOPT --
>>> WARNING - Allowing Override for key TARGET_ARCH
>>
>>> PROGRESS - --Running UnitTestFrameworkPkg: Host Unit Test Compiler
>>> Plugin NOOPT --
>>> WARNING - Allowing Override for key TARGET_ARCH
>>
>> - The below seems to be related to the "spell checking in audit
>> mode" known issues at
>> <https://github.com/tianocore/edk2/blob/master/.pytool/Readme.md>.
>>
>> What does "audit mode" mean?
>
> see readme here.
> https://github.com/tianocore/edk2/tree/master/.pytool/Plugin/SpellCheck#configuration
>
>
> Basically runs the tests and reports all the errors as warnings and then
> reports the test as skipped.
> It was hopefully to encourage package maintainers to fix their spelling
> issues but not breaking the build on day 1.
>
>
>>
>> FWIW, the packages listed below (EmulatorPkg, MdeModulePkg,
>> MdePkg, NetworkPkg, OvmfPkg, ShellPkg, UefiCpuPkg) is a proper
>> subset of the packges noted in ".pytool/Readme.md" -- the latter
>> mentions CryptoPkg and SecurityPkg in addition.
>>
>>> PROGRESS - --Running EmulatorPkg: Spell Check Test NO-TARGET --
>>> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET
>>
>>> PROGRESS - --Running MdeModulePkg: Spell Check Test NO-TARGET --
>>> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET
>>
>>> PROGRESS - --Running MdePkg: Spell Check Test NO-TARGET --
>>> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET
>>
>>> PROGRESS - --Running NetworkPkg: Spell Check Test NO-TARGET --
>>> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET
>>
>>> PROGRESS - --Running OvmfPkg: Spell Check Test NO-TARGET --
>>> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET
>>
>>> PROGRESS - --Running ShellPkg: Spell Check Test NO-TARGET --
>>> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET
>>
>>> PROGRESS - --Running UefiCpuPkg: Spell Check Test NO-TARGET --
>>> WARNING - --->Test Skipped: in plugin! Spell Check Test NO-TARGET
>>
>> - Not sure about the last one:
>>
>>> PROGRESS - --Running FatPkg: Library Class Check Test NO-TARGET --
>>> WARNING - --->Test Skipped: in plugin! Library Class Check Test
>>> NO-TARGET
>>
>>
>> * One suggestion (in addition to the others made in this thread):
>>
>> - Please add "BaseToolsBuild" to "BaseTools/.gitignore".
>
> There is a bugzilla for this already.
> https://bugzilla.tianocore.org/show_bug.cgi?id=2581
>
>
>>
>>
>> * A question:
>>
>> - On github, there is some logic that restricts the full Core CI build
>> to packages that may be affected by a patch series. Is this available
>> locally?
>>
>> According to the Core CI documentation, the "stuart_ci_build" command
>> can be limited with "-p" / "-a" / "-t" flags; given a commit range,
>> how do we calculate the "tightest" flag values?
>
> A couple things here.
> You can always look at the azurepipelines files to get a better
> understanding of the CI process. Hopefully there is no magic (besides
> being able to read yaml). You can see here:
> https://github.com/tianocore/edk2/blob/master/.azurepipelines/templates/pr-gate-steps.yml#L35
>
>
> It is actually another stuart tool.
> stuart_pr_eval.
> We have bug to document it.
> https://github.com/tianocore/edk2-pytool-extensions/issues/89
> But the python file is pretty well documented. There are 4 policies that
> can trigger the need for a change to build a package.
> See here
> https://github.com/tianocore/edk2-pytool-extensions/blob/master/edk2toolext/invocables/edk2_pr_eval.py#L117
>
>
>
>
>>
>> Thank you very much!
>> Laszlo
>>
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#64551): https://edk2.groups.io/g/devel/message/64551
Mute This Topic: https://groups.io/mt/76285051/1813853
Group Owner: devel+owner at edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [edk2-devel-archive at redhat.com]
-=-=-=-=-=-=-=-=-=-=-=-
More information about the edk2-devel-archive
mailing list