Re: 回复: 回复: 回复: [edk2-devel] [PATCH v1 1/1] Add MemoryFence implementation for RiscV64

Daniel Schaefer daniel.schaefer at hpe.com
Wed Jun 2 02:35:38 UTC 2021 

On 6/2/21 10:16 AM, gaoliming wrote:
> Laszlo:
>    Thanks for the detail information about QemuFwCfgLib. So, this library is arch generic. Now, it is only consumed by ARM and AARCH64.
> 
> Daniel:
>    Can you show more on how RiscV64 uses this QemuFwCfgLib?

See below. QEMU for the generic RISCV64 platform is implemented almost 
like Arm with device tree and fw-cfg device, quite unlike X64 which uses 
port-mapped I/O and doesn't have device trees.

> Thanks
> Liming
>> -----邮件原件-----
>> 发件人: devel at edk2.groups.io <devel at edk2.groups.io> 代表 Laszlo Ersek
>> 发送时间: 2021年6月1日 15:58
>> 收件人: gaoliming <gaoliming at byosoft.com.cn>; devel at edk2.groups.io;
>> daniel.schaefer at hpe.com; ardb at kernel.org
>> 抄送: 'Chang, Abner (HPS SW/FW Technologist)' <abner.chang at hpe.com>;
>> 'Michael D Kinney' <michael.d.kinney at intel.com>; 'Zhiguang Liu'
>> <zhiguang.liu at intel.com>; 'Leif Lindholm' <leif at nuviainc.com>
>> 主题: Re: 回复: 回复: [edk2-devel] [PATCH v1 1/1] Add MemoryFence
>> implementation for RiscV64
>>
>> On 06/01/21 02:56, gaoliming wrote:
>>> Seemly, Edk2\ArmVirtPkg\Library\QemuFwCfgLib\QemuFwCfgLib.inf is not
>> arch
>>> specific library. It can also be used in RISCV64.
>>>
>>>
>>>
>>> Ard and Laszlo:
>>>
>>>   If ArmVirtPkg\Library\QemuFwCfgLib is arch generic, can it be moved
>> from
>>> ArmVirtPkg into OvmfPkg?
>>
>> ArmVirtPkg/Library/QemuFwCfgLib is a QemuFwCfgLib instance that is
>> currently only used by the ArmVirtQemu and ArmVirtQemuKernel platforms.

​Yes, but we're in the process of Creating RiscvVirtPkg that just like 
ArmVirtPkg supports the generic "virt" machine type of qemu-system-riscv64.

>> It depends on the FDT_CLIENT_PROTOCOL, from
>> "ArmVirtPkg/ArmVirtPkg.dec"
>> and "ArmVirtPkg/Include/Protocol/FdtClient.h", to locate the fw_cfg
>> device. The protocol is ArmVirtPkg specific. Due to the protocol depex,
>> the library is also DXE_DRIVER and UEFI_DRIVER only.

​Yes, we also use that in our RiscvVirtPkg ;)

>> The library uses the MMIO data registers of the fw_cfg device by
>> default; if the DMA interface is supported, then it uses the DMA
>> interface. In both cases, some registers are accessed with 64-bit
>> accesses if MDE_CPU_AARCH64 is defined, and with 32-bit accesses
>> otherwise.
>>
>> I don't see how RISCV could reuse this library verbatim.

​Why not? We can boot perfectly fine from virtio (which is PCI) with 
these lines in our RIscvVirt.dsc:

# Flattened Device Tree (FDT) access library
FdtLib|EmbeddedPkg/Library/FdtLib/FdtLib.inf

# PCI Libraries
PciLib|MdePkg/Library/BasePciLibPciExpress/BasePciLibPciExpress.inf
PciExpressLib|ArmVirtPkg/Library/BaseCachingPciExpressLib/BaseCachingPciExpressLib.inf
PciExpressLib|MdePkg/Library/BasePciExpressLib/BasePciExpressLib.inf
PciCapLib|OvmfPkg/Library/BasePciCapLib/BasePciCapLib.inf
PciCapPciSegmentLib|OvmfPkg/Library/BasePciCapPciSegmentLib/BasePciCapPciSegmentLib.inf
PciCapPciIoLib|OvmfPkg/Library/UefiPciCapPciIoLib/UefiPciCapPciIoLib.inf

# Virtio Support
VirtioLib|OvmfPkg/Library/VirtioLib/VirtioLib.inf
VirtioMmioDeviceLib|OvmfPkg/Library/VirtioMmioDeviceLib/VirtioMmioDeviceLib.inf
QemuFwCfgLib|ArmVirtPkg/Library/QemuFwCfgLib/QemuFwCfgLib.inf
QemuFwCfgS3Lib|OvmfPkg/Library/QemuFwCfgS3Lib/BaseQemuFwCfgS3LibNull.inf
QemuFwCfgSimpleParserLib|OvmfPkg/Library/QemuFwCfgSimpleParserLib/QemuFwCfgSimpleParserLib.inf
QemuLoadImageLib|OvmfPkg/Library/GenericQemuLoadImageLib/GenericQemuLoadImageLib.inf

# PCI support
PciPcdProducerLib|ArmVirtPkg/Library/FdtPciPcdProducerLib/FdtPciPcdProducerLib.inf
PciSegmentLib|MdePkg/Library/BasePciSegmentLibPci/BasePciSegmentLibPci.inf
PciHostBridgeLib|ArmVirtPkg/Library/FdtPciHostBridgeLib/FdtPciHostBridgeLib.inf
PciHostBridgeUtilityLib|OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.inf

Excerpt from the edk2 boot log:

InvalidateInstructionCacheRange:RISC-V unsupported function.
Loading driver at 0x000BF7EC000 EntryPoint=0x000BF7EC28A 
PciHostBridgeDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF BF31EF18
ProtectUefiImageCommon - 0xBF334BC0
   - 0x00000000BF7EC000 - 0x0000000000014080
PROGRESS CODE: V03040002 I0
Found FwCfg @ 0x10100008/0x10100000
Found FwCfg DMA @ 0x10100010
ProcessPciHost: Config[0x30000000+0x10000000) Bus[0x0..0xFF] 
Io[0x0+0x10000)@0x3000000 Mem32[0x40000000+0x40000000)@00

​If you look at the source code of the QEMU implementation of riscv64 
virt and compare it to aarch64 they are very similar, but quite 
different from x64.

>> The linked patch at
>> <https://github.com/riscv/riscv-edk2/commit/8c7960ef860c65f2646912c3dc
>> cbb308a>
>> is a no-go; the MDE_CPU_RISCV64 macro has no place in an ArmVirtPkg
>> library.
>>
>> The library can be moved to the new directory
>>
>>    OvmfPkg/Library/DxeQemuFwCfgLibFdtMmio
>>
>> (note the rename in the last pathname component), but it needs to be
>> done in multiple steps. The FDT protocol GUID and structure definition
>> has to be moved at first, separately from the library, and every move
>> operation (i.e., each one of the protocol move and the library muve)
>> must be implemented with *at least* three steps -- copy the original to
>> OvmfPkg (updating BASE_NAME at once), update DSC references under
>> ArmVirtPkg, remove the original under ArmVirtPkg. Only then can you add
>> customizations.

Okay, I understand and agree. We'll think about all of the Arm modules 
and libraries we're currently using, how to move those and send an RFC 
soon. We have not encountered any other issues, other than this change, 
when using the Arm code.

>> Regarding the processor type macros, I believe Mike recently introduced
>> ISA-independent macros, for expressing 64-bit vs. 32-bit. I'm not
>> exactly sure about the details, but I think we now have a macro under
>> MdePkg that says "64-bit processor" without having to state AARCH64 or
>> RISCV64.

I found some at
CryptoPkg/Library/Include/CrtLibSupport.h
and
RedfishPkg/PrivateInclude/Library/RedfishCrtLib.h
called SIXTY_FOUR_BIT and THIRTY_TWO_BIT.

Not sure if they are for general consumption?
It seems that Mike suggests those to be defined in the platform's DSC?
See d0bf83e1cca

>> Thanks
>> Laszlo
>>
>>
>>>
>>>
>>>
>>> Thanks
>>>
>>> Liming
>>>
>>> 发件人: devel at edk2.groups.io <devel at edk2.groups.io> 代表 Daniel
>> Schaefer
>>> 发送时间: 2021年5月21日 20:46
>>> 收件人: devel at edk2.groups.io; gaoliming at byosoft.com.cn
>>> 抄送: Chang, Abner (HPS SW/FW Technologist) <abner.chang at hpe.com>;
>> 'Michael
>>> D Kinney' <michael.d.kinney at intel.com>; 'Zhiguang Liu'
>>> <zhiguang.liu at intel.com>; 'Leif Lindholm' <leif at nuviainc.com>
>>> 主题: Re: 回复: [edk2-devel] [PATCH v1 1/1] Add MemoryFence
>> implementation
>>> for RiscV64
>>>
>>>
>>>
>>> It's not required to go into that tag.
>>>
>>> We need two more patches that we haven't submitted yet to boot on
>> Qemu.
>>>
>>>
>>>
>>> Would it be okay if we used a library from ArmVirtPkg for RISCV64?
>>>
>>> See:
>>>
>> https://github.com/riscv/riscv-edk2/commit/8c7960ef860c65f2646912c3dcc
>> bb308a
>>> 98e0cc3
>>>
>>> Or does it have to be moved to some other place first?
>>>
>>>    _____
>>>
>>> From: devel at edk2.groups.io <mailto:devel at edk2.groups.io>
>>> <devel at edk2.groups.io <mailto:devel at edk2.groups.io> > on behalf of
>> gaoliming
>>> <gaoliming at byosoft.com.cn <mailto:gaoliming at byosoft.com.cn> >
>>> Sent: Friday, May 21, 2021 14:35
>>> To: devel at edk2.groups.io <mailto:devel at edk2.groups.io>
>>> <devel at edk2.groups.io <mailto:devel at edk2.groups.io> >; Schaefer, Daniel
>>> <daniel.schaefer at hpe.com <mailto:daniel.schaefer at hpe.com> >
>>> Cc: Chang, Abner (HPS SW/FW Technologist) <abner.chang at hpe.com
>>> <mailto:abner.chang at hpe.com> >; 'Michael D Kinney'
>>> <michael.d.kinney at intel.com <mailto:michael.d.kinney at intel.com> >;
>> 'Zhiguang
>>> Liu' <zhiguang.liu at intel.com <mailto:zhiguang.liu at intel.com> >; 'Leif
>>> Lindholm' <leif at nuviainc.com <mailto:leif at nuviainc.com> >
>>> Subject: 回复: 回复: [edk2-devel] [PATCH v1 1/1] Add MemoryFence
>>> implementation for RiscV64
>>>
>>>
>>>
>>> Daniel:
>>>
>>>   Thanks for your information. Acked-by: Liming Gao
>> <gaoliming at byosoft.com.cn
>>> <mailto:gaoliming at byosoft.com.cn> >
>>>
>>>
>>>
>>>    And, do you request to merge this patch for edk2 stable tag 202105?
>>>
>>>
>>>
>>> Thanks
>>>
>>> Liming
>>>
>>> 发件人: devel at edk2.groups.io <mailto:devel at edk2.groups.io>
>>> <devel at edk2.groups.io <mailto:devel at edk2.groups.io> > 代表 Daniel
>> Schaefer
>>> 发送时间: 2021年5月21日 13:27
>>> 收件人: devel at edk2.groups.io <mailto:devel at edk2.groups.io> ;
>>> gaoliming at byosoft.com.cn <mailto:gaoliming at byosoft.com.cn>
>>> 抄送: Chang, Abner (HPS SW/FW Technologist) <abner.chang at hpe.com
>>> <mailto:abner.chang at hpe.com> >; 'Michael D Kinney'
>>> <michael.d.kinney at intel.com <mailto:michael.d.kinney at intel.com> >;
>> 'Zhiguang
>>> Liu' <zhiguang.liu at intel.com <mailto:zhiguang.liu at intel.com> >; 'Leif
>>> Lindholm' <leif at nuviainc.com <mailto:leif at nuviainc.com> >
>>> 主题: Re: 回复: [edk2-devel] [PATCH v1 1/1] Add MemoryFence
>> implementation
>>> for RiscV64
>>>
>>>
>>>
>>> Great!
>>>
>>>
>>>
>>> It is verified I can boot Linux from a virtio ESP using this patch on QEMU
>>> virt machine.
>>>
>>> See:
>>>
>> https://github.com/riscv/riscv-edk2-platforms/runs/2618819010?check_suite
>> _fo
>>> cus=true
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Daniel
>>>
>>>    _____
>>>
>>> From: devel at edk2.groups.io <mailto:devel at edk2.groups.io>
>>> <devel at edk2.groups.io <mailto:devel at edk2.groups.io> > on behalf of
>> gaoliming
>>> <gaoliming at byosoft.com.cn <mailto:gaoliming at byosoft.com.cn> >
>>> Sent: Friday, May 21, 2021 13:14
>>> To: devel at edk2.groups.io <mailto:devel at edk2.groups.io>
>>> <devel at edk2.groups.io <mailto:devel at edk2.groups.io> >; Schaefer, Daniel
>>> <daniel.schaefer at hpe.com <mailto:daniel.schaefer at hpe.com> >
>>> Cc: Chang, Abner (HPS SW/FW Technologist) <abner.chang at hpe.com
>>> <mailto:abner.chang at hpe.com> >; 'Michael D Kinney'
>>> <michael.d.kinney at intel.com <mailto:michael.d.kinney at intel.com> >;
>> 'Zhiguang
>>> Liu' <zhiguang.liu at intel.com <mailto:zhiguang.liu at intel.com> >; 'Leif
>>> Lindholm' <leif at nuviainc.com <mailto:leif at nuviainc.com> >
>>> Subject: 回复: 回复: [edk2-devel] [PATCH v1 1/1] Add MemoryFence
>>> implementation for RiscV64
>>>
>>>
>>>
>>> Daniel:
>>>    Now, it is clear to me. So, I suggest to merge this change when it is
>>> verified on generic RISC-V QEMU virt machine. Is it OK?
>>>
>>> Thanks
>>> Liming
>>>> -----邮件原件-----
>>>> 发件人: devel at edk2.groups.io <mailto:devel at edk2.groups.io>
>>> <devel at edk2.groups.io <mailto:devel at edk2.groups.io> > 代表 Daniel
>>>> Schaefer
>>>> 发送时间: 2021年5月18日 10:35
>>>> 收件人: devel at edk2.groups.io <mailto:devel at edk2.groups.io> ;
>>> gaoliming at byosoft.com.cn <mailto:gaoliming at byosoft.com.cn>
>>>> 抄送: 'Abner Chang' <abner.chang at hpe.com
>> <mailto:abner.chang at hpe.com> >;
>>> 'Michael D Kinney'
>>>> <michael.d.kinney at intel.com <mailto:michael.d.kinney at intel.com> >;
>>> 'Zhiguang Liu' <zhiguang.liu at intel.com <mailto:zhiguang.liu at intel.com> >;
>>> 'Leif
>>>> Lindholm' <leif at nuviainc.com <mailto:leif at nuviainc.com> >
>>>> 主题: Re: 回复: [edk2-devel] [PATCH v1 1/1] Add MemoryFence
>>>> implementation for RiscV64
>>>>
>>>> On 5/18/21 9:04 AM, gaoliming wrote:
>>>>> Daniel:
>>>>>   Seemly, this API is missing in BaseLib for RiscV64 arch. How do you
>>> detect
>>>>> this issue?
>>>>
>>>> What do you mean it's missing?
>>>> Yes MemoryFence() for RiscV64 is missing currently, that's why I'm adding
>>> it
>>>> here.
>>>>
>>>> Maybe you mean that it's not currently used? That's also true.
>>>> I'm enabling the generic QEMU virt machine (like OVMF or ArmVirtPkg) for
>>>> RISC-V.
>>>> At least QemuFwCfgLib and VirtioLib need it.
>>>> That's why I have the need to add this implementation now.
>>>>
>>>> Does that clear it up?
>>>>
>>>>> Thanks
>>>>> Liming
>>>>>> -----邮件原件-----
>>>>>> 发件人: devel at edk2.groups.io <mailto:devel at edk2.groups.io>
>>> <devel at edk2.groups.io <mailto:devel at edk2.groups.io> > 代表 Daniel
>>>>>> Schaefer
>>>>>> 发送时间: 2021年5月16日 2:13
>>>>>> 收件人: devel at edk2.groups.io <mailto:devel at edk2.groups.io>
>>>>>> 抄送: Abner Chang <abner.chang at hpe.com
>> <mailto:abner.chang at hpe.com> >;
>>> Michael D Kinney
>>>>>> <michael.d.kinney at intel.com <mailto:michael.d.kinney at intel.com> >;
>>> Liming Gao <gaoliming at byosoft.com.cn
>> <mailto:gaoliming at byosoft.com.cn> >;
>>>>>> Zhiguang Liu <zhiguang.liu at intel.com
>> <mailto:zhiguang.liu at intel.com> >;
>>> Leif Lindholm
>>> <leif at nuviainc.com <mailto:leif at nuviainc.com> >
>>>>>> 主题: [edk2-devel] [PATCH v1 1/1] Add MemoryFence implementation
>> for
>>>>>> RiscV64
>>>>>>
>>>>>> Cc: Abner Chang <abner.chang at hpe.com
>> <mailto:abner.chang at hpe.com> >
>>>>>> Cc: Michael D Kinney <michael.d.kinney at intel.com
>>> <mailto:michael.d.kinney at intel.com> >
>>>>>> Cc: Liming Gao <gaoliming at byosoft.com.cn
>> <mailto:gaoliming at byosoft.com.
>>> cn> >
>>>>>> Cc: Zhiguang Liu <zhiguang.liu at intel.com
>>> <mailto:zhiguang.liu at intel.com> >
>>>>>> Cc: Leif Lindholm <leif at nuviainc.com <mailto:leif at nuviainc.com> >
>>>>>> Signed-off-by: Daniel Schaefer <daniel.schaefer at hpe.com
>> <mailto:daniel.
>>> schaefer at hpe.com> >
>>>>>> ---
>>>>>>   MdePkg/Library/BaseLib/BaseLib.inf           |  1 +
>>>>>>   MdePkg/Library/BaseLib/RiscV64/MemoryFence.S | 33
>>>>>> ++++++++++++++++++++
>>>>>>   2 files changed, 34 insertions(+)
>>>>>>
>>>>>> diff --git a/MdePkg/Library/BaseLib/BaseLib.inf
>>>>>> b/MdePkg/Library/BaseLib/BaseLib.inf
>>>>>> index b76f3af380ea..b7ab5f632366 100644
>>>>>> --- a/MdePkg/Library/BaseLib/BaseLib.inf
>>>>>> +++ b/MdePkg/Library/BaseLib/BaseLib.inf
>>>>>> @@ -399,6 +399,7 @@
>>>>>>     RiscV64/DisableInterrupts.c
>>>>>>
>>>>>>
>>>>>>     RiscV64/EnableInterrupts.c
>>>>>>
>>>>>>
>>>>>>     RiscV64/CpuPause.c
>>>>>>
>>>>>>
>>>>>> +  RiscV64/MemoryFence.S             | GCC
>>>>>>
>>>>>>
>>>>>>     RiscV64/RiscVSetJumpLongJump.S    | GCC
>>>>>>
>>>>>>
>>>>>>     RiscV64/RiscVCpuBreakpoint.S      | GCC
>>>>>>
>>>>>>
>>>>>>     RiscV64/RiscVCpuPause.S           | GCC
>>>>>>
>>>>>>
>>>>>> diff --git a/MdePkg/Library/BaseLib/RiscV64/MemoryFence.S
>>>>>> b/MdePkg/Library/BaseLib/RiscV64/MemoryFence.S
>>>>>> new file mode 100644
>>>>>> index 000000000000..283df9356a9a
>>>>>> --- /dev/null
>>>>>> +++ b/MdePkg/Library/BaseLib/RiscV64/MemoryFence.S
>>>>>> @@ -0,0 +1,33 @@
>>>>>>
>>>>>
>>> +##-------------------------------------------------------------------------
>>>>> -----
>>>>>>
>>>>>>
>>>>>> +#
>>>>>>
>>>>>>
>>>>>> +# MemoryFence() for RiscV64
>>>>>>
>>>>>>
>>>>>> +
>>>>>>
>>>>>>
>>>>>> +# Copyright (c) 2021, Hewlett Packard Enterprise Development. All
>>> rights
>>>>>> reserved.
>>>>>>
>>>>>>
>>>>>> +#
>>>>>>
>>>>>>
>>>>>> +# SPDX-License-Identifier: BSD-2-Clause-Patent
>>>>>>
>>>>>>
>>>>>> +#
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>> +##-------------------------------------------------------------------------
>>>>> -----
>>>>>>
>>>>>>
>>>>>> +
>>>>>>
>>>>>>
>>>>>> +.text
>>>>>>
>>>>>>
>>>>>> +.p2align 2
>>>>>>
>>>>>>
>>>>>> +
>>>>>>
>>>>>>
>>>>>> +ASM_GLOBAL ASM_PFX(MemoryFence)
>>>>>>
>>>>>>
>>>>>> +
>>>>>>
>>>>>>
>>>>>> +
>>>>>>
>>>>>>
>>>>>> +#/**
>>>>>>
>>>>>>
>>>>>> +#  Used to serialize load and store operations.
>>>>>>
>>>>>>
>>>>>> +#
>>>>>>
>>>>>>
>>>>>> +#  All loads and stores that proceed calls to this function are
>>>>> guaranteed to
>>>>>> be
>>>>>>
>>>>>>
>>>>>> +#  globally visible when this function returns.
>>>>>>
>>>>>>
>>>>>> +#
>>>>>>
>>>>>>
>>>>>> +#**/
>>>>>>
>>>>>>
>>>>>> +#VOID
>>>>>>
>>>>>>
>>>>>> +#EFIAPI
>>>>>>
>>>>>>
>>>>>> +#MemoryFence (
>>>>>>
>>>>>>
>>>>>> +#  VOID
>>>>>>
>>>>>>
>>>>>> +#  );
>>>>>>
>>>>>>
>>>>>> +#
>>>>>>
>>>>>>
>>>>>> +ASM_PFX(MemoryFence):
>>>>>>
>>>>>>
>>>>>> +    // Fence on all memory and I/O
>>>>>>
>>>>>>
>>>>>> +    fence
>>>>>>
>>>>>>
>>>>>> +    ret
>>>>>>
>>>>>>
>>>>>> --
>>>>>> 2.30.1
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>>
> 
> 
> 
> 
> 
> 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#75929): https://edk2.groups.io/g/devel/message/75929
Mute This Topic: https://groups.io/mt/83250003/1813853
Group Owner: devel+owner at edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [edk2-devel-archive at redhat.com]
-=-=-=-=-=-=-=-=-=-=-=-

More information about the edk2-devel-archive mailing list