[edk2-devel] [edk2-libc Patch 1/1] AppPkg\Applications\Python\Python-3.6.8: GCC tool chain support
Michael D Kinney
michael.d.kinney at intel.com
Tue May 31 23:45:12 UTC 2022
Reviewed-by: Michael D Kinney <michael.d.kinney at intel.com>
I do see that there are 3 documentation files in Python-3.6.8.
One is from the Python project. The other 2 look like they both
contain EDK II build instructions. Perhaps these should be combined into
a single file for EDK II builds for all supported tool chains and archs.
Mike
> -----Original Message-----
> From: devel at edk2.groups.io <devel at edk2.groups.io> On Behalf Of Jayaprakash, N
> Sent: Wednesday, May 18, 2022 11:42 AM
> To: devel at edk2.groups.io
> Subject: [edk2-devel] [edk2-libc Patch 1/1] AppPkg\Applications\Python\Python-3.6.8: GCC tool chain support
>
> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3779
>
> This commit contains the changes made to Python interpreter
> 3.6.8 to cross compile using the GCC tool chain on Ubuntu 20.04 LTS
> system for UEFI. The ctypes module has been excluded from GCC compilation as it requires
> additional porting work to use GCC tool chain support for UEFI environment.
> Added GCC Compilation steps as BKM document to facilitate the environment setup and
> compilation on Linux using GCC tool chain. Besides this provided a shell script to
> generate the usable Python UEFI package with all the dependencies taken from the
> build environment.
>
> Cc: Rebecca Cran <rebecca at nuviainc.com>
> Cc: Michael D Kinney <michael.d.kinney at intel.com>
> Signed-off-by: Jayaprakash N <n.jayaprakash at intel.com>
> ---
> .../Python-3.6.8/GCCCompilationBKMs.rst | 193 ++++++++++++++++++
> .../PyMod-3.6.8/Include/pyconfig.h | 2 +-
> .../Python-3.6.8/PyMod-3.6.8/Modules/config.c | 7 +-
> .../PyMod-3.6.8/Modules/getpath.c | 2 +-
> .../Python/Python-3.6.8/Python368.inf | 21 +-
> .../Python-3.6.8/create_python368_pkg.sh | 100 +++++++++
> 6 files changed, 312 insertions(+), 13 deletions(-)
> create mode 100644 AppPkg/Applications/Python/Python-3.6.8/GCCCompilationBKMs.rst
> create mode 100644 AppPkg/Applications/Python/Python-3.6.8/create_python368_pkg.sh
>
> diff --git a/AppPkg/Applications/Python/Python-3.6.8/GCCCompilationBKMs.rst b/AppPkg/Applications/Python/Python-
> 3.6.8/GCCCompilationBKMs.rst
> new file mode 100644
> index 0000000..ed5fcbe
> --- /dev/null
> +++ b/AppPkg/Applications/Python/Python-3.6.8/GCCCompilationBKMs.rst
> @@ -0,0 +1,193 @@
> +1. Introduction
> +---------------
> +This is a Best-Known Methods document capturing the details of build environment setup for compiling the Python UEFI
> +interpreter version 3.6.8 using GCC tool chain on Linux based operating systems. These BKMs have been tested with WSL2
> +(Windows Subsystem for Linux 2) based Ubuntu 20.04 LTS environment on Windows 10. Only 64-bit builds are possible due
> +to a limitation in the LibC support for GCC compiler tool chain on UEFI. If you are working on a Ubuntu 20.04 LTS system
> +you may skip the sections 1.1 and 1.2
> +
> +
> +1.1. WSL2 Installation on Windows 10 OS
> +----------------------------------------
> +
> +The WSL2 environment can be setup by following the instructions given in the below webpage
> +https://pureinfotech.com/install-windows-subsystem-linux-2-windows-10/
> +
> +1.2. Installing Ubuntu 20.04 LTS on WSL2 on Windows 10 OS
> +----------------------------------------------------------
> +
> +The instructions provided in the below webpage will help in installing the Ubuntu 20.04 LTS OS environment on WSL2 on Windows 10
> OS.
> +https://www.altisconsulting.com/au/insights/installing-ubuntu-bash-for-windows-10-wsl2-setup/
> +
> +2. EDK2 build environment setup
> +-------------------------------
> +
> +You may follow the instructions provided in the below webpage to setup the edk2 build environment.
> +
> +https://github.com/tianocore/tianocore.github.io/wiki/Using-EDK-II-with-Native-GCC
> +
> +For the purposes of describing the BKMs, we will be using the following paths.
> +Edk2 source tree: $HOME/src/edk2
> +Native GCC version >=4.x compiler installation: /usr/bin/gcc
> +Intel ASL Compiler installation: /usr/bin/iasl
> +
> +
> +2.1. Installing essential packages for GCC compiler
> +----------------------------------------------------
> +
> +Install the below required essential packages to facilitate compilation of the edk2 repo using GCC compiler tool chain.
> +Several Ubuntu packages will be needed to set up the build environment for EDK II.
> +
> +The following command will install all required packages::
> +
> + bash$ sudo apt-get update
> + bash$ sudo apt install build-essential
> + bash$ sudo apt install uuid-dev iasl git nasm python-is-python3
> +
> +Package - Description
> + * build-essential : Informational list of build-essential packages
> + * uuid-dev : Universally Unique ID library (headers and static libraries)
> + * iasl : Intel ASL compiler/decompiler (also provided by acpica-tools)
> + * git : Support for git revision control system
> + * nasm : General-purpose x86 assembler
> + * python-is-python3 : Ubuntu 20.04 python command is 'python3' but edk2 tools use 'python'
> +
> +Once the above packages are installed, then follow the instructions in the webpage to setup the rest of the build
> +environment
> +https://github.com/tianocore/tianocore.github.io/wiki/Common-instructions
> +
> +
> +2.2. Cloning edk2 git repo
> +---------------------------
> +
> +Get the edk2 source tree using Git commands.
> +Execute the below commands to clone the edk2 git repo::
> + bash$ mkdir ~/src
> + bash$ cd ~/src
> + bash$ git clone https://github.com/tianocore/edk2
> +
> +This will create a folder named edk2 as a clone of the edk2 git repo.
> +Execute the below commands to checkout / initialize all the git submodules::
> + bash$ cd ~/src/edk2
> + bash$ git submodule update --init
> +
> +The above command initializes all the submodules in the git repo
> +The below commands help to setup the environment variables required for building the edk2 repo::
> +
> + bash$ make -C BaseTools
> + bash$ . edksetup.sh
> +
> +Run the below make command from ~/src folder i.e change working directory to ~/src and run the make command as given below ::
> +
> + bash$ cd ~/src
> + bash$ make -C edk2/BaseTools
> +
> +Setup build shell environment ::
> +
> + bash$ cd ~/src/edk2
> + bash$ export EDK_TOOLS_PATH=$HOME/src/edk2/BaseTools
> + bash$ . edksetup.sh BaseTools
> +
> +
> +Set Build Target Information
> +For the Conf/target.txt file, find the following lines:
> +ACTIVE_PLATFORM = Nt32Pkg/Nt32Pkg.dsc
> +
> +TOOL_CHAIN_TAG = MYTOOLS
> +
> +And change the corresponding lines to match these:
> +
> +ACTIVE_PLATFORM = MdeModulePkg/MdeModulePkg.dsc
> +
> +TOOL_CHAIN_TAG = GCC5
> +
> +Install GCC5 on Ubuntu 20.04 LTS by downloading it from
> +https://askubuntu.com/questions/1235819/ubuntu-20-04-gcc-version-lower-than-gcc-7
> +
> +This requires latest version of the nasm tool to be installed.
> +The nasm latest version >= 2.15.05
> +
> +Debian package for nasm 2.15.05 is available at
> +
> +https://ubuntu.pkgs.org/21.10/ubuntu-universe-amd64/nasm_2.15.05-1_amd64.deb.html
> +
> +Download it using the wget command as given below::
> +
> + bash$ sudo wget http://archive.ubuntu.com/ubuntu/pool/universe/n/nasm/nasm_2.15.05-1_amd64.deb
> +
> +
> +Install the the above downloaded nasm debian package using the following command::
> +
> + bash$ sudo dpkg -i nasm_2.15.05-1_amd64.deb
> +
> +2.3. Install additional packages required for building edk2 with GCC
> +---------------------------------------------------------------------
> +
> +Install additional package required for building edk2 repo::
> +
> + bash$ sudo apt install libx11-dev
> + bash$ sudo apt install libxext-dev
> +
> +
> +2.4 Clone edk2-libc and copy the contents to edk2
> +--------------------------------------------------
> +
> +Now clone the edk2-libc git repo by following / executing the below commands::
> +
> + bash$ cd ~/src
> + bash$ git clone https://github.com/tianocore/edk2-libc.git
> + bash$ export PACKAGES_PATH=$HOME/src/edk2:$HOME/src/edk2-libc
> +
> +
> +Verify that you can build the hello world application from AppPkg under edk2-libc by running the following commands
> +Build hello.inf from edk2-libc applications::
> +
> + $bash build -p AppPkg/AppPkg.dsc -m AppPkg/Applications/Hello/Hello.inf
> +
> +Copy the contents of edk2-libc to edk2 folder::
> +
> + bash$ cp -rf ~/src/edk2-libc/* ~/src/edk2/
> +
> +2.5. Build Python Interpreter using GCC
> +----------------------------------------
> +Uncomment the line containing following inf file name in AppPkg/AppPkg.dsc file
> +AppPkg/Applications/Python/Python-3.6.8/Python368.inf
> +
> +Execute the below command to build the X64 version of the Python 3.6.8 interpreter using GCC tool chain::
> +
> + bash$ cd AppPkg/Applications/Python/Python-3.6.8/
> + bash$ python srcprep.py
> + bash$ cd ~/src/edk2
> + bash$ build -a X64 -b RELEASE -p AppPkg/AppPkg.dsc -m AppPkg/Applications/Python/Python-3.6.8/Python368.inf
> + or
> + bash$ build -a X64 -b RELEASE -p AppPkg/AppPkg.dsc
> +
> +
> +2.5 Create Python UEFI package with all dependencies
> +-----------------------------------------------------
> +
> +To create an usable Python UEFI package with all the dependencies from the build environment,
> +you can use the bash shell script create_python368_pkg.sh available under /AppPkg/Applications/Python/Python-3.6.8
> +folder.
> +
> +Use the following commands to create Python UEFI package::
> +
> + bash$ cd ~/src/edk2/AppPkg/Applications/Python/Python-3.6.8/
> + bash$ chmod 777 create_python368_pkg.sh
> + ./create_python368_pkg.sh GCC5 RELEASE X64 myPyUEFI
> +
> +where GCC5 refers to the tool chain, RELEASE refers to the build type, X64 refers to the architecture
> +myPyUEFI refers to the name of the folder where the Python UEFI package needs to be created.
> +
> +When you run the create_python368_pkg.sh without any parameters it lists the help
> +information as shown below:
> +
> +Shell Script to create Python EFI Package.
> +
> +Usage: ./create_python368_pkg.sh <ToolChain> <Target> <Architecture> <OutFolder>
> +
> +Where
> + * ToolChain : name of the tool chain such as GCC5
> + * Target : build type such as RELEASE, DEBUG
> + * Architecture : Architecture such as X64
> + * OutFolder : Output directory for creating the package
> diff --git a/AppPkg/Applications/Python/Python-3.6.8/PyMod-3.6.8/Include/pyconfig.h b/AppPkg/Applications/Python/Python-
> 3.6.8/PyMod-3.6.8/Include/pyconfig.h
> index d4685da..9d08982 100644
> --- a/AppPkg/Applications/Python/Python-3.6.8/PyMod-3.6.8/Include/pyconfig.h
> +++ b/AppPkg/Applications/Python/Python-3.6.8/PyMod-3.6.8/Include/pyconfig.h
> @@ -870,7 +870,7 @@
> #undef HAVE_THREAD_H
>
> /* Define to 1 if you have the 'timegm' function. */
> -#undef HAVE_TIMEGM
> +#define HAVE_TIMEGM 1
>
> /* Define to 1 if you have the 'times' function. */
> #undef HAVE_TIMES
> diff --git a/AppPkg/Applications/Python/Python-3.6.8/PyMod-3.6.8/Modules/config.c b/AppPkg/Applications/Python/Python-
> 3.6.8/PyMod-3.6.8/Modules/config.c
> index 0e7f65e..7217a80 100644
> --- a/AppPkg/Applications/Python/Python-3.6.8/PyMod-3.6.8/Modules/config.c
> +++ b/AppPkg/Applications/Python/Python-3.6.8/PyMod-3.6.8/Modules/config.c
> @@ -83,7 +83,10 @@ extern PyObject* PyInit__stat(void);
> extern PyObject* PyInit__opcode(void);
> extern PyObject* PyInit_faulthandler(void);
> // _ctypes
> +#if defined(UEFI_MSVC_32) || defined(UEFI_MSVC_64)
> extern PyObject* PyInit__ctypes(void);
> +#endif
> +
> extern PyObject* init_sqlite3(void);
>
> // EfiPy
> @@ -155,7 +158,9 @@ struct _inittab _PyImport_Inittab[] = {
> {"_string", PyInit__string},
> {"_stat", PyInit__stat},
> {"_opcode", PyInit__opcode},
> - { "_ctypes", PyInit__ctypes },
> +#if defined(UEFI_MSVC_32) || defined(UEFI_MSVC_64)
> + { "_ctypes", PyInit__ctypes },
> +#endif
> /* Sentinel */
> {0, 0}
> };
> diff --git a/AppPkg/Applications/Python/Python-3.6.8/PyMod-3.6.8/Modules/getpath.c b/AppPkg/Applications/Python/Python-
> 3.6.8/PyMod-3.6.8/Modules/getpath.c
> index ad10784..7ee4d5e 100644
> --- a/AppPkg/Applications/Python/Python-3.6.8/PyMod-3.6.8/Modules/getpath.c
> +++ b/AppPkg/Applications/Python/Python-3.6.8/PyMod-3.6.8/Modules/getpath.c
> @@ -112,7 +112,7 @@
> #endif
>
>
> -#define SIFY_I( x ) L#x
> +#define SIFY_I( x ) L"" #x
> #define SIFY( y ) SIFY_I( y )
>
> #ifndef PREFIX
> diff --git a/AppPkg/Applications/Python/Python-3.6.8/Python368.inf b/AppPkg/Applications/Python/Python-3.6.8/Python368.inf
> index 4ab0cd3..207b907 100644
> --- a/AppPkg/Applications/Python/Python-3.6.8/Python368.inf
> +++ b/AppPkg/Applications/Python/Python-3.6.8/Python368.inf
> @@ -251,23 +251,24 @@ Modules/zlib
> Modules/zlib/zutil.c #
>
> #Modules/ctypes
> - PyMod-$(PYTHON_VERSION)/Modules/_ctypes/_ctypes.c #
> - Modules/_ctypes/stgdict.c #
> - Modules/_ctypes/libffi_msvc/prep_cif.c #
> - PyMod-$(PYTHON_VERSION)/Modules/_ctypes/malloc_closure.c #
> - PyMod-$(PYTHON_VERSION)/Modules/_ctypes/libffi_msvc/ffi.c #
> - Modules/_ctypes/cfield.c #
> - PyMod-$(PYTHON_VERSION)/Modules/_ctypes/callproc.c #
> - Modules/_ctypes/callbacks.c #
> + PyMod-$(PYTHON_VERSION)/Modules/_ctypes/_ctypes.c | MSFT #
> + Modules/_ctypes/stgdict.c | MSFT #
> + Modules/_ctypes/libffi_msvc/prep_cif.c | MSFT#
> + PyMod-$(PYTHON_VERSION)/Modules/_ctypes/malloc_closure.c | MSFT #
> + PyMod-$(PYTHON_VERSION)/Modules/_ctypes/libffi_msvc/ffi.c | MSFT #
> + Modules/_ctypes/cfield.c | MSFT #
> + PyMod-$(PYTHON_VERSION)/Modules/_ctypes/callproc.c | MSFT #
> + Modules/_ctypes/callbacks.c | MSFT #
>
> [Sources.IA32]
> - Modules/_ctypes/libffi_msvc/win32.c #
> + Modules/_ctypes/libffi_msvc/win32.c | MSFT #
>
> [Sources.X64]
> - Modules/_ctypes/libffi_msvc/win64.asm #
> + Modules/_ctypes/libffi_msvc/win64.asm | MSFT #
>
> [BuildOptions]
> MSFT:*_*_*_CC_FLAGS = /GL- /Oi- /wd4018 /wd4054 /wd4055 /wd4101 /wd4131 /wd4152 /wd4204 /wd4210 /wd4244 /wd4267 /wd4305
> /wd4310 /wd4389 /wd4701 /wd4702 /wd4706 /wd4456 /wd4312 /wd4457 /wd4459 /wd4474 /wd4476
> /I$(WORKSPACE)\AppPkg\Applications\Python\Python-3.6.8\Include /DHAVE_MEMMOVE /DUSE_PYEXPAT_CAPI /DXML_STATIC -D UEFI /WX-
> /DXML_POOR_ENTROPY /DUEFI_C_SOURCE
> + GCC:*_*_*_CC_FLAGS = -Wno-unused-function -Wno-format -Wno-error -fno-strict-aliasing -
> I$(WORKSPACE)/AppPkg/Applications/Python/Python-3.6.8/Include -DHAVE_MEMMOVE -DUSE_PYEXPAT_CAPI -DXML_STATIC -DXML_POOR_ENTROPY
>
> [BuildOptions.IA32]
> MSFT:*_*_*_CC_FLAGS = /DUEFI_MSVC_32
> diff --git a/AppPkg/Applications/Python/Python-3.6.8/create_python368_pkg.sh b/AppPkg/Applications/Python/Python-
> 3.6.8/create_python368_pkg.sh
> new file mode 100644
> index 0000000..f529fa4
> --- /dev/null
> +++ b/AppPkg/Applications/Python/Python-3.6.8/create_python368_pkg.sh
> @@ -0,0 +1,100 @@
> +#!/bin/bash
> +
> +TOOL_CHAIN_TAG=$1
> +TARGET=$2
> +ARCH=$3
> +OUT_FOLDER=$4
> +
> +function usage {
> + echo
> + echo Shell Script to create Python EFI Package.
> + echo
> + echo
> + echo "Usage: $0 <ToolChain> <Target> <Architecture> <OutFolder>"
> + echo "Where "
> + echo " ToolChain : name of the tool chain such as GCC5"
> + echo " Target : build type such as RELEASE, DEBUG"
> + echo " Architecture : Architecture such as X64"
> + echo " OutFolder : Output directory for creating the package"
> + echo
> +}
> +
> +function error {
> + echo Failed to Create Python EFI Package
> + echo Python368.efi application is not available at "Build/AppPkg/$TARGET"_"$TOOL_CHAIN_TAG/$ARCH/"
> + echo Follow the instructions in Py368ReadMe.txt to build Python interpreter
> + echo Then use this script to create a Python EFI package
> +}
> +
> +
> +# Validate the ToolChain input
> +if [ -z $TOOL_CHAIN_TAG ]
> +then
> + echo Invalid command line arguments passed, please see the below usage instructions
> + usage
> + exit
> +fi
> +
> +# Validate the Target input
> +if [ -z $TARGET ]
> +then
> + echo Invalid command line arguments passed, please see the below usage instructions
> + usage
> + exit
> +fi
> +
> +# Validate the Architecture input
> +if [ -z $ARCH ]
> +then
> + echo Invalid command line arguments passed, please see the below usage instructions
> + usage
> + exit
> +fi
> +
> +# Validate the OutFolder input
> +if [ -z $OUT_FOLDER ]
> +then
> + echo Invalid command line arguments passed, please see the below usage instructions
> + usage
> + exit
> +fi
> +
> +cd ../../../../
> +PYTHON_BIN="Build/AppPkg/$TARGET"_"$TOOL_CHAIN_TAG/$ARCH/Python368.efi"
> +if [ ! -f $PYTHON_BIN ]
> +then
> + error
> + exit
> +fi
> +
> +
> +# create /EFI/Tools folder in the Output folder if it doesn't exists
> +if [ ! -d $OUT_FOLDER/EFI/Tools ]
> +then
> + mkdir -p $OUT_FOLDER/EFI/Tools
> +fi
> +
> +cp -f $PYTHON_BIN $OUT_FOLDER/EFI/Tools/
> +
> +if [ ! -d $OUT_FOLDER/EFI/StdLib/lib/python36.8 ]
> +then
> + mkdir -p $OUT_FOLDER/EFI/StdLib/lib/python36.8
> +fi
> +
> +if [ ! -d $OUT_FOLDER/EFI/StdLib/etc ]
> +then
> + mkdir -p $OUT_FOLDER/EFI/StdLib/etc
> +fi
> +
> +cp -rf AppPkg/Applications/Python/Python-3.6.8/Lib/* $OUT_FOLDER/EFI/StdLib/lib/python36.8/
> +cp -rf StdLib/Efi/StdLib/etc/* $OUT_FOLDER/EFI/StdLib/etc/
> +
> +if [ ${OUT_FOLDER:0:1} == "/" ]
> +then
> + echo Python EFI package available at $OUT_FOLDER
> +else
> + echo Python EFI package available at `pwd`/$OUT_FOLDER
> +fi
> +
> +
> +
> --
> 2.33.0.windows.1
>
>
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#90118): https://edk2.groups.io/g/devel/message/90118
Mute This Topic: https://groups.io/mt/91193074/1813853
Group Owner: devel+owner at edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [edk2-devel-archive at redhat.com]
-=-=-=-=-=-=-=-=-=-=-=-
More information about the edk2-devel-archive
mailing list