[edk2-devel] [PATCH] BaseTools: Change non-ascii character of StructurePcd comment

Thu Apr 29 02:12:40 UTC 2021

So this tool converts UNI file contents into DSC file contents?

The fix only addresses a single character issue for (R), which means we will keep getting errors.

Perhaps we should convert all non-ASCII characters to '?'.  If the developer using the tool
does not like the '?' characters, then they can update the strings in the UNI file to
only use ASCII characters.

BTW...I did run into this type of issue a lot when converting documents to GitBook markdown.
I recall adding many special character conversion rules to get to ASCII markdown files.
This is why I know this approach has many problems.

Mike

> -----Original Message-----
> From: Chen, Christine <yuwei.chen at intel.com>
> Sent: Wednesday, April 28, 2021 6:24 PM
> To: gaoliming <gaoliming at byosoft.com.cn>; devel at edk2.groups.io; Kinney, Michael D <michael.d.kinney at intel.com>
> Cc: Feng, Bob C <bob.c.feng at intel.com>
> Subject: RE: [edk2-devel] [PATCH] BaseTools: Change non-ascii character of StructurePcd comment
> 
> Thanks for reminder~  : )
> Thus personally think if do not want the non-ascii character in StructurePcd dsc file, the remove or change
> operation should be covered by ConvertFceToStructurePcd.py
> 
> Thanks,
> Christine
> > -----Original Message-----
> > From: gaoliming <gaoliming at byosoft.com.cn>
> > Sent: Thursday, April 29, 2021 9:18 AM
> > To: devel at edk2.groups.io; Chen, Christine <yuwei.chen at intel.com>; Kinney,
> > Michael D <michael.d.kinney at intel.com>
> > Cc: Feng, Bob C <bob.c.feng at intel.com>
> > Subject: 回复: [edk2-devel] [PATCH] BaseTools: Change non-ascii character of
> > StructurePcd comment
> >
> > Christine:
> >   FCE prints HII question information that is from HII driver UNI file. It may
> > have non-ascii character.
> >
> > Thanks
> > Liming
> > > -----邮件原件-----
> > > 发件人: devel at edk2.groups.io <devel at edk2.groups.io> 代表 Yuwei Chen
> > > 发送时间: 2021年4月29日 8:25
> > > 收件人: Kinney, Michael D <michael.d.kinney at intel.com>;
> > > devel at edk2.groups.io
> > > 抄送: Feng, Bob C <bob.c.feng at intel.com>; Liming Gao
> > > <gaoliming at byosoft.com.cn>
> > > 主题: Re: [edk2-devel] [PATCH] BaseTools: Change non-ascii character of
> > > StructurePcd comment
> > >
> > > Hi Mike,
> > >
> > > The StructurePcd dsc file generated by our tool will have the
> > > non-ASCII character.
> > > The input file of ConvertFceToStructurePcd.py is generated by FCE
> > > tool, which has the circle R non-ASCII character. This patch change
> > > this character to ACSII character when using
> > > ConvertFceToStructurePcd.py to generate the StructurePcd dsc file.
> > >
> > > Best Regards,
> > > Christine
> > >
> > > > -----Original Message-----
> > > > From: Kinney, Michael D <michael.d.kinney at intel.com>
> > > > Sent: Thursday, April 29, 2021 12:10 AM
> > > > To: devel at edk2.groups.io; Chen, Christine <yuwei.chen at intel.com>;
> > > Kinney,
> > > > Michael D <michael.d.kinney at intel.com>
> > > > Cc: Feng, Bob C <bob.c.feng at intel.com>; Liming Gao
> > > > <gaoliming at byosoft.com.cn>
> > > > Subject: RE: [edk2-devel] [PATCH] BaseTools: Change non-ascii
> > > > character of StructurePcd comment
> > > >
> > > > What file type contains the non-ASCII character?
> > > >
> > > > I would prefer to see the source file with non ASCII character be
> > > > updated instead of building this conversion into the tools.
> > > >
> > > > Mike
> > > >
> > > > > -----Original Message-----
> > > > > From: devel at edk2.groups.io <devel at edk2.groups.io> On Behalf Of
> > > > > Yuwei Chen
> > > > > Sent: Wednesday, April 28, 2021 1:45 AM
> > > > > To: devel at edk2.groups.io
> > > > > Cc: Feng, Bob C <bob.c.feng at intel.com>; Liming Gao
> > > > > <gaoliming at byosoft.com.cn>
> > > > > Subject: [edk2-devel] [PATCH] BaseTools: Change non-ascii
> > > > > character of StructurePcd comment
> > > > >
> > > > > Currently, the ConvertFceToStructurePcd.py tool generate
> > > > > StructurePcd dsc file with comments including non-ascii character
> > > > > circle R. This patch changes the non-ascii character circle R to
> > > > > (R) when adding the comment.
> > > > >
> > > > > Cc: Bob Feng <bob.c.feng at intel.com>
> > > > > Cc: Liming Gao <gaoliming at byosoft.com.cn>
> > > > > Signed-off-by: Yuwei Chen <yuwei.chen at intel.com>
> > > > > ---
> > > > >  BaseTools/Scripts/ConvertFceToStructurePcd.py | 4 ++++
> > > > >  1 file changed, 4 insertions(+)
> > > > >
> > > > > diff --git a/BaseTools/Scripts/ConvertFceToStructurePcd.py
> > > > > b/BaseTools/Scripts/ConvertFceToStructurePcd.py
> > > > > index 2052db8c4b..d029ed6a28 100644
> > > > > --- a/BaseTools/Scripts/ConvertFceToStructurePcd.py
> > > > > +++ b/BaseTools/Scripts/ConvertFceToStructurePcd.py
> > > > > @@ -285,6 +285,10 @@ class Config(object):
> > > > >          comment_list = value_re.findall(line) # the string \\... in "Q...."
> > > line
> > > > >          comment_list[0] = comment_list[0].replace('//', '')
> > > > >          comment = comment_list[0].strip()
> > > > > +        comment_b = bytes(comment, encoding = "utf8")
> > > > > +        if b"\xae" in comment_b:
> > > > > +            comment_b = comment_b.replace(b"\xc2\xae", b"(R)") #
> > > Change
> > > > the circle "R" character to ascii character
> > > > > +        comment = str(comment_b, encoding = "utf-8")
> > > > >          line=value_re.sub('',line) #delete \\... in "Q...." line
> > > > >          list1=line.split(' ')
> > > > >          value=self.value_parser(list1)
> > > > > --
> > > > > 2.26.1.windows.1
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > >
> > >
> > >
> > > 
> > >
> >
> >

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#74593): https://edk2.groups.io/g/devel/message/74593
Mute This Topic: https://groups.io/mt/82445891/1813853
Group Owner: devel+owner at edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [edk2-devel-archive at redhat.com]
-=-=-=-=-=-=-=-=-=-=-=-