From mihajlov at linux.vnet.ibm.com Tue Nov 5 09:03:30 2013 From: mihajlov at linux.vnet.ibm.com (Viktor Mihajlovski) Date: Tue, 5 Nov 2013 10:03:30 +0100 Subject: [Libvirt-cim] [RESEND PATCH 0/3] libvirt-cim: Fix provider registration Message-ID: <1383642213-10260-1-git-send-email-mihajlov@linux.vnet.ibm.com> The following series fixes two issues in the provider registration process of the libvirt-cim RPM package. First, the RPM scriptlets didn't properly reflect the invocation sequence during an RPM upgrade. This is fixed by the first patch. Further, the registration using wildcards (*.mof) failed because of an inter-MOF depedency. Fixed by the second patch. Finally, the class deletion sequence in provider-register.sh was prone to failure, resulting in residual entries in the Pegasus class repository. Addressed by third patch. Note: since the redhat.com mailing lists are bouncing my emails I am sending from a different account temporarily until this is resolved. Viktor Mihajlovski (3): build: Fix incorrect provider registration in upgrade path build: Fix provider registration issues schema: Fix class removal with Pegasus libvirt-cim.spec.in | 282 ++++++++++++++++++++++++++++++++++++++++---------- provider-register.sh | 8 +- 2 files changed, 231 insertions(+), 59 deletions(-) -- 1.7.9.5 From mihajlov at linux.vnet.ibm.com Tue Nov 5 09:03:33 2013 From: mihajlov at linux.vnet.ibm.com (Viktor Mihajlovski) Date: Tue, 5 Nov 2013 10:03:33 +0100 Subject: [Libvirt-cim] [PATCH 3/3] schema: Fix class removal with Pegasus In-Reply-To: <1383642213-10260-1-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1383642213-10260-1-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <1383642213-10260-4-git-send-email-mihajlov@linux.vnet.ibm.com> In provider de-registration step, the provider-register.sh script is attempting to delete all libvirt-cim classes from the Pegasus repository. Pegasus refuses to delete classes if it still has child classes in the repository. While the MOF files are processed in reverse order, the classes were still deleted in their original order, which can fail due to inter-class dependencies. Changed to reverse the class deletion order on a per MOF file base. Signed-off-by: Viktor Mihajlovski --- provider-register.sh | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/provider-register.sh b/provider-register.sh index b730ef3..abe8e95 100755 --- a/provider-register.sh +++ b/provider-register.sh @@ -332,8 +332,12 @@ pegasus_uninstall() echo "Error: wbemexec not found" >&2 return 1 fi - CLASSES=`cat $mymofs 2> /dev/null | grep '^class'| cut -d ' ' -f 2 | uniq` - + for mof in $mymofs + do + # We must delete the classes in reverse order per MOF file + MOFCLASSES=`cat $mof 2> /dev/null | grep '^[[:space:]]*class' | sed 's/ \+/ /g' | tac | cut -d ' ' -f 2` + CLASSES="$CLASSES $MOFCLASSES" + done for _TEMPDIR in /var/tmp /tmp do if test -w $_TEMPDIR -- 1.7.9.5 From mihajlov at linux.vnet.ibm.com Tue Nov 5 09:03:31 2013 From: mihajlov at linux.vnet.ibm.com (Viktor Mihajlovski) Date: Tue, 5 Nov 2013 10:03:31 +0100 Subject: [Libvirt-cim] [PATCH 1/3] build: Fix incorrect provider registration in upgrade path In-Reply-To: <1383642213-10260-1-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1383642213-10260-1-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <1383642213-10260-2-git-send-email-mihajlov@linux.vnet.ibm.com> The scriplet logic was incorrectly assuming that the superseded package's %postun script would be called before the superceding packages %pre/%post. This effectively broke upgrades of libvirt-cim because all the providers were deregistered. We are now checking whether we are in an upgrade situation and if so will not deregister the providers in postun. Another enhancement is that we do a full deregistration in the %post section for tog-pegasus now. This should make installs and upgrades more robust against potentially damaged repositories (e.g., on development systems). As a reminder here's a short description of RPM's scriptlet processing. action: install upgrade uninstall %pre 1 >1 - %post 1 >1 - %preun - 1 0 %postun - 1 0 Scriptlet invocation order on upgrade 1. %pre(new_package) 2. %post(new_package) 3. %preun(old_package) 4. %postun(old_package) Signed-off-by: Viktor Mihajlovski --- libvirt-cim.spec.in | 115 ++++++++++++++++++++++++++++++--------------------- 1 file changed, 68 insertions(+), 47 deletions(-) diff --git a/libvirt-cim.spec.in b/libvirt-cim.spec.in index 459650c..b50cbd1 100644 --- a/libvirt-cim.spec.in +++ b/libvirt-cim.spec.in @@ -64,7 +64,6 @@ mkdir -p $RPM_BUILD_ROOT at INFO_STORE@ %clean rm -fr $RPM_BUILD_ROOT -%pre %define REGISTRATION %{_datadir}/%{name}/*.registration %define SCHEMA %{_datadir}/%{name}/*.mof @@ -77,24 +76,46 @@ rm -fr $RPM_BUILD_ROOT %define CIMV2_REG %{_datadir}/%{name}/{HostedResourcePool,ElementCapabilities,HostedService,HostedDependency,ElementConformsToProfile,HostedAccessPoint}.registration %define CIMV2_MOF %{_datadir}/%{name}/{HostedResourcePool,ElementCapabilities,HostedService,HostedDependency,RegisteredProfile,ComputerSystem,ElementConformsToProfile,HostedAccessPoint}.mof +%pre # _If_ there is already a version of this installed, we must deregister # the classes we plan to install in post, otherwise we may corrupt # the pegasus repository. This is convention in other provider packages -%{_datadir}/%{name}/provider-register.sh -d -t pegasus \ - -n @CIM_VIRT_NS@ \ - -r %{REGISTRATION} -m %{SCHEMA} >/dev/null 2>&1 || true +if [ $1 -gt 1 ] +then + if [ -x /usr/sbin/cimserver ] + then + %{_datadir}/%{name}/provider-register.sh -d -t pegasus \ + -n @CIM_VIRT_NS@ \ + -r %{REGISTRATION} -m %{SCHEMA} >/dev/null 2>&1 || true + %{_datadir}/%{name}/provider-register.sh -d -t pegasus \ + -n root/interop \ + -r %{INTEROP_REG} -m %{INTEROP_MOF} -v >/dev/null 2>&1 || true + %{_datadir}/%{name}/provider-register.sh -d -t pegasus \ + -n root/PG_InterOp \ + -r %{PGINTEROP_REG} -m %{PGINTEROP_MOF} -v >/dev/null 2>&1 || true + %{_datadir}/%{name}/provider-register.sh -d -t pegasus \ + -n root/cimv2\ + -r %{CIMV2_REG} -m %{CIMV2_MOF} -v >/dev/null 2>&1 || true + fi # Remove open-pegasus-specific providers installed in sfcb repository # by older libvirt-cim packages -%{_datadir}/%{name}/provider-register.sh -d -t sfcb \ - -n root/PG_InterOp \ - -r %{PGINTEROP_REG} -m %{PGINTEROP_MOF} >/dev/null 2>&1 || true - + if [ -x /usr/sbin/sfcbd ] + then + %{_datadir}/%{name}/provider-register.sh -d -t sfcb \ + -n root/PG_InterOp \ + -r %{PGINTEROP_REG} -m %{PGINTEROP_MOF} >/dev/null 2>&1 || true + fi +fi %post /sbin/ldconfig -%{_datadir}/%{name}/install_base_schema.sh %{_datadir}/%{name} +if [ $1 -eq 1 ] +then +# Install the CIM base schema if this is the initial install + %{_datadir}/%{name}/install_base_schema.sh %{_datadir}/%{name} +fi %if 0%{?fedora} >= 17 || 0%{?rhel} >= 7 if [ "`systemctl is-active tog-pegasus.service`" = "active" ] @@ -112,65 +133,65 @@ rm -fr $RPM_BUILD_ROOT if [ -x /usr/sbin/cimserver ] then -%{_datadir}/%{name}/provider-register.sh -t pegasus \ + %{_datadir}/%{name}/provider-register.sh -t pegasus \ -n @CIM_VIRT_NS@ \ -r %{REGISTRATION} -m %{SCHEMA} >/dev/null 2>&1 || true -%{_datadir}/%{name}/provider-register.sh -t pegasus \ - -n @CIM_VIRT_NS@ \ - -r %{REGISTRATION} -m %{SCHEMA} >/dev/null 2>&1 || true -%{_datadir}/%{name}/provider-register.sh -t pegasus \ + %{_datadir}/%{name}/provider-register.sh -t pegasus \ -n root/interop \ -r %{INTEROP_REG} -m %{INTEROP_MOF} -v >/dev/null 2>&1 || true -%{_datadir}/%{name}/provider-register.sh -t pegasus \ + %{_datadir}/%{name}/provider-register.sh -t pegasus \ -n root/PG_InterOp \ -r %{PGINTEROP_REG} -m %{PGINTEROP_MOF} -v >/dev/null 2>&1 || true -%{_datadir}/%{name}/provider-register.sh -t pegasus \ + %{_datadir}/%{name}/provider-register.sh -t pegasus \ -n root/cimv2\ -r %{CIMV2_REG} -m %{CIMV2_MOF} -v >/dev/null 2>&1 || true fi if [ -x /usr/sbin/sfcbd ] then -%{_datadir}/%{name}/provider-register.sh -t sfcb \ - -n root/virt \ - -r %{REGISTRATION} -m %{SCHEMA} >/dev/null 2>&1 || true -%{_datadir}/%{name}/provider-register.sh -t sfcb \ + %{_datadir}/%{name}/provider-register.sh -t sfcb \ -n root/virt \ -r %{REGISTRATION} -m %{SCHEMA} >/dev/null 2>&1 || true -%{_datadir}/%{name}/provider-register.sh -t sfcb \ + %{_datadir}/%{name}/provider-register.sh -t sfcb \ -n root/interop \ -r %{INTEROP_REG} -m %{INTEROP_MOF} -v >/dev/null 2>&1 || true -%{_datadir}/%{name}/provider-register.sh -t sfcb \ + %{_datadir}/%{name}/provider-register.sh -t sfcb \ -n root/cimv2\ -r %{CIMV2_REG} -m %{CIMV2_MOF} -v >/dev/null 2>&1 || true fi %preun -if [ -x /usr/sbin/cimserver ] +# The uninstall scriptlets are called after the install scriptlets +# in the upgrade case. Therefore we must only deregister the providers +# when $1 == 0 (final remove). +if [ $1 -eq 0 ] then -%{_datadir}/%{name}/provider-register.sh -d -t pegasus \ - -n root/virt \ - -r %{REGISTRATION} -m %{SCHEMA} >/dev/null 2>&1 || true -%{_datadir}/%{name}/provider-register.sh -d -t pegasus \ - -n root/interop \ - -r %{INTEROP_REG} -m %{INTEROP_MOF} >/dev/null 2>&1 || true -%{_datadir}/%{name}/provider-register.sh -d -t pegasus \ - -n root/PG_InterOp \ - -r %{PGINTEROP_REG} -m %{PGINTEROP_MOF} >/dev/null 2>&1 || true -%{_datadir}/%{name}/provider-register.sh -d -t pegasus \ - -n root/cimv2 \ - -r %{CIMV2_REG} -m %{CIMV2_MOF} >/dev/null 2>&1 || true -fi -if [ -x /usr/sbin/sfcbd ] -then -%{_datadir}/%{name}/provider-register.sh -d -t sfcb \ - -n root/virt \ - -r %{REGISTRATION} -m %{SCHEMA} >/dev/null 2>&1 || true -%{_datadir}/%{name}/provider-register.sh -d -t sfcb \ - -n root/interop \ - -r %{INTEROP_REG} -m %{INTEROP_MOF} >/dev/null 2>&1 || true -%{_datadir}/%{name}/provider-register.sh -d -t sfcb \ - -n root/cimv2 \ - -r %{CIMV2_REG} -m %{CIMV2_MOF} >/dev/null 2>&1 || true + if [ -x /usr/sbin/cimserver ] + then + %{_datadir}/%{name}/provider-register.sh -d -t pegasus \ + -n root/virt \ + -r %{REGISTRATION} -m %{SCHEMA} >/dev/null 2>&1 || true + %{_datadir}/%{name}/provider-register.sh -d -t pegasus \ + -n root/interop \ + -r %{INTEROP_REG} -m %{INTEROP_MOF} >/dev/null 2>&1 || true + %{_datadir}/%{name}/provider-register.sh -d -t pegasus \ + -n root/PG_InterOp \ + -r %{PGINTEROP_REG} -m %{PGINTEROP_MOF} >/dev/null 2>&1 || true + %{_datadir}/%{name}/provider-register.sh -d -t pegasus \ + -n root/cimv2 \ + -r %{CIMV2_REG} -m %{CIMV2_MOF} >/dev/null 2>&1 || true + fi + if [ -x /usr/sbin/sfcbd ] + then + %{_datadir}/%{name}/provider-register.sh -d -t sfcb \ + -n root/virt \ + -r %{REGISTRATION} -m %{SCHEMA} >/dev/null 2>&1 || true + %{_datadir}/%{name}/provider-register.sh -d -t sfcb \ + -n root/interop \ + -r %{INTEROP_REG} -m %{INTEROP_MOF} >/dev/null 2>&1 || true + %{_datadir}/%{name}/provider-register.sh -d -t sfcb \ + -n root/cimv2 \ + -r %{CIMV2_REG} -m %{CIMV2_MOF} >/dev/null 2>&1 || true + fi fi %postun -p /sbin/ldconfig -- 1.7.9.5 From mihajlov at linux.vnet.ibm.com Tue Nov 5 09:03:32 2013 From: mihajlov at linux.vnet.ibm.com (Viktor Mihajlovski) Date: Tue, 5 Nov 2013 10:03:32 +0100 Subject: [Libvirt-cim] [PATCH 2/3] build: Fix provider registration issues In-Reply-To: <1383642213-10260-1-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1383642213-10260-1-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <1383642213-10260-3-git-send-email-mihajlov@linux.vnet.ibm.com> This commit addresses an issue with provider registration during RPM install or update. The schema registration by wildcard doesn't take into consideration that there are dependencies between the MOF files leading to a partially populated repository and a not working libvirt-cim provider. Fixed by explicitly stating the mof and registration files in the necessary order. Further a minor false error message coming from the systemd service detection was bound to cause irritation. This is suppressed now. Signed-off-by: Viktor Mihajlovski --- libvirt-cim.spec.in | 167 ++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 157 insertions(+), 10 deletions(-) diff --git a/libvirt-cim.spec.in b/libvirt-cim.spec.in index b50cbd1..c96451b 100644 --- a/libvirt-cim.spec.in +++ b/libvirt-cim.spec.in @@ -64,17 +64,164 @@ mkdir -p $RPM_BUILD_ROOT at INFO_STORE@ %clean rm -fr $RPM_BUILD_ROOT -%define REGISTRATION %{_datadir}/%{name}/*.registration -%define SCHEMA %{_datadir}/%{name}/*.mof +%define REGISTRATION %{_datadir}/%{name}/ComputerSystem.registration \\\ + %{_datadir}/%{name}/LogicalDisk.registration \\\ + %{_datadir}/%{name}/NetworkPort.registration \\\ + %{_datadir}/%{name}/Memory.registration \\\ + %{_datadir}/%{name}/Processor.registration \\\ + %{_datadir}/%{name}/SystemDevice.registration \\\ + %{_datadir}/%{name}/VSSD.registration \\\ + %{_datadir}/%{name}/HostSystem.registration \\\ + %{_datadir}/%{name}/HostedDependency.registration \\\ + %{_datadir}/%{name}/VirtualSystemManagementService.registration \\\ + %{_datadir}/%{name}/VirtualSystemManagementCapabilities.registration \\\ + %{_datadir}/%{name}/EnabledLogicalElementCapabilities.registration \\\ + %{_datadir}/%{name}/AllocationCapabilities.registration \\\ + %{_datadir}/%{name}/SettingsDefineCapabilities.registration \\\ + %{_datadir}/%{name}/MemoryPool.registration \\\ + %{_datadir}/%{name}/ElementCapabilities.registration \\\ + %{_datadir}/%{name}/ProcessorPool.registration \\\ + %{_datadir}/%{name}/DiskPool.registration \\\ + %{_datadir}/%{name}/HostedResourcePool.registration \\\ + %{_datadir}/%{name}/ComputerSystemIndication.registration \\\ + %{_datadir}/%{name}/ResourceAllocationSettingDataIndication.registration \\\ + %{_datadir}/%{name}/SwitchService.registration \\\ + %{_datadir}/%{name}/ComputerSystemMigrationIndication.registration \\\ + %{_datadir}/%{name}/ResourceAllocationSettingData.registration \\\ + %{_datadir}/%{name}/ResourcePoolConfigurationService.registration \\\ + %{_datadir}/%{name}/ResourcePoolConfigurationCapabilities.registration \\\ + %{_datadir}/%{name}/VSSDComponent.registration \\\ + %{_datadir}/%{name}/SettingsDefineState.registration \\\ + %{_datadir}/%{name}/NetPool.registration \\\ + %{_datadir}/%{name}/ResourceAllocationFromPool.registration \\\ + %{_datadir}/%{name}/ElementAllocatedFromPool.registration \\\ + %{_datadir}/%{name}/HostedService.registration \\\ + %{_datadir}/%{name}/ElementSettingData.registration \\\ + %{_datadir}/%{name}/VSMigrationCapabilities.registration \\\ + %{_datadir}/%{name}/VSMigrationService.registration \\\ + %{_datadir}/%{name}/ElementConformsToProfile.registration \\\ + %{_datadir}/%{name}/VSMigrationSettingData.registration \\\ + %{_datadir}/%{name}/VirtualSystemSnapshotService.registration \\\ + %{_datadir}/%{name}/VirtualSystemSnapshotServiceCapabilities.registration \\\ + %{_datadir}/%{name}/ConcreteComponent.registration \\\ + %{_datadir}/%{name}/ConsoleRedirectionService.registration \\\ + %{_datadir}/%{name}/ConsoleRedirectionServiceCapabilities.registration \\\ + %{_datadir}/%{name}/ServiceAffectsElement.registration \\\ + %{_datadir}/%{name}/KVMRedirectionSAP.registration \\\ + %{_datadir}/%{name}/DisplayController.registration \\\ + %{_datadir}/%{name}/PointingDevice.registration \\\ + %{_datadir}/%{name}/GraphicsPool.registration \\\ + %{_datadir}/%{name}/InputPool.registration \\\ + %{_datadir}/%{name}/HostedAccessPoint.registration \\\ + %{_datadir}/%{name}/ServiceAccessBySAP.registration \\\ + %{_datadir}/%{name}/SAPAvailableForElement.registration \\\ + %{_datadir}/%{name}/FilterEntry.registration \\\ + %{_datadir}/%{name}/FilterList.registration \\\ + %{_datadir}/%{name}/EntriesInFilterList.registration \\\ + %{_datadir}/%{name}/NestedFilterList.registration \\\ + %{_datadir}/%{name}/AppliedFilterList.registration \\\ + %{_datadir}/%{name}/HostedFilterList.registration -%define INTEROP_REG %{_datadir}/%{name}/{RegisteredProfile,ElementConformsToProfile,ReferencedProfile}.registration -%define INTEROP_MOF %{_datadir}/%{name}/{ComputerSystem,HostSystem,RegisteredProfile,DiskPool,MemoryPool,NetPool,ProcessorPool,VSMigrationService,ElementConformsToProfile,ReferencedProfile,AllocationCapabilities}.mof +%define SCHEMA %{_datadir}/%{name}/ComputerSystem.mof \\\ + %{_datadir}/%{name}/LogicalDisk.mof \\\ + %{_datadir}/%{name}/NetworkPort.mof \\\ + %{_datadir}/%{name}/Memory.mof \\\ + %{_datadir}/%{name}/Processor.mof \\\ + %{_datadir}/%{name}/SystemDevice.mof \\\ + %{_datadir}/%{name}/Virt_VSSD.mof \\\ + %{_datadir}/%{name}/VSSD.mof \\\ + %{_datadir}/%{name}/HostSystem.mof \\\ + %{_datadir}/%{name}/HostedDependency.mof \\\ + %{_datadir}/%{name}/VirtualSystemManagementService.mof \\\ + %{_datadir}/%{name}/VirtualSystemManagementCapabilities.mof \\\ + %{_datadir}/%{name}/EnabledLogicalElementCapabilities.mof \\\ + %{_datadir}/%{name}/AllocationCapabilities.mof \\\ + %{_datadir}/%{name}/SettingsDefineCapabilities.mof \\\ + %{_datadir}/%{name}/MemoryPool.mof \\\ + %{_datadir}/%{name}/ElementCapabilities.mof \\\ + %{_datadir}/%{name}/ProcessorPool.mof \\\ + %{_datadir}/%{name}/DiskPool.mof \\\ + %{_datadir}/%{name}/HostedResourcePool.mof \\\ + %{_datadir}/%{name}/RegisteredProfile.mof \\\ + %{_datadir}/%{name}/ElementConformsToProfile.mof \\\ + %{_datadir}/%{name}/ComputerSystemIndication.mof \\\ + %{_datadir}/%{name}/ResourceAllocationSettingDataIndication.mof \\\ + %{_datadir}/%{name}/SwitchService.mof \\\ + %{_datadir}/%{name}/ComputerSystemMigrationIndication.mof \\\ + %{_datadir}/%{name}/Virt_ResourceAllocationSettingData.mof \\\ + %{_datadir}/%{name}/ResourceAllocationSettingData.mof \\\ + %{_datadir}/%{name}/ResourcePoolConfigurationService.mof \\\ + %{_datadir}/%{name}/ResourcePoolConfigurationCapabilities.mof \\\ + %{_datadir}/%{name}/VSSDComponent.mof \\\ + %{_datadir}/%{name}/SettingsDefineState.mof \\\ + %{_datadir}/%{name}/NetPool.mof \\\ + %{_datadir}/%{name}/ResourceAllocationFromPool.mof \\\ + %{_datadir}/%{name}/ElementAllocatedFromPool.mof \\\ + %{_datadir}/%{name}/HostedService.mof \\\ + %{_datadir}/%{name}/ElementSettingData.mof \\\ + %{_datadir}/%{name}/VSMigrationCapabilities.mof \\\ + %{_datadir}/%{name}/VSMigrationService.mof \\\ + %{_datadir}/%{name}/VSMigrationSettingData.mof \\\ + %{_datadir}/%{name}/VirtualSystemSnapshotService.mof \\\ + %{_datadir}/%{name}/VirtualSystemSnapshotServiceCapabilities.mof \\\ + %{_datadir}/%{name}/ConcreteComponent.mof \\\ + %{_datadir}/%{name}/ConsoleRedirectionService.mof \\\ + %{_datadir}/%{name}/ConsoleRedirectionServiceCapabilities.mof \\\ + %{_datadir}/%{name}/ServiceAffectsElement.mof \\\ + %{_datadir}/%{name}/KVMRedirectionSAP.mof \\\ + %{_datadir}/%{name}/DisplayController.mof \\\ + %{_datadir}/%{name}/PointingDevice.mof \\\ + %{_datadir}/%{name}/GraphicsPool.mof \\\ + %{_datadir}/%{name}/InputPool.mof \\\ + %{_datadir}/%{name}/HostedAccessPoint.mof \\\ + %{_datadir}/%{name}/ServiceAccessBySAP.mof \\\ + %{_datadir}/%{name}/SAPAvailableForElement.mof \\\ + %{_datadir}/%{name}/FilterEntry.mof \\\ + %{_datadir}/%{name}/FilterList.mof \\\ + %{_datadir}/%{name}/EntriesInFilterList.mof \\\ + %{_datadir}/%{name}/NestedFilterList.mof \\\ + %{_datadir}/%{name}/AppliedFilterList.mof \\\ + %{_datadir}/%{name}/HostedFilterList.mof -%define PGINTEROP_REG %{_datadir}/%{name}/{RegisteredProfile,ElementConformsToProfile,ReferencedProfile}.registration -%define PGINTEROP_MOF %{_datadir}/%{name}/{RegisteredProfile,ElementConformsToProfile,ReferencedProfile}.mof +%define INTEROP_REG %{_datadir}/%{name}/RegisteredProfile.registration \\\ + %{_datadir}/%{name}/ElementConformsToProfile.registration \\\ + %{_datadir}/%{name}/ReferencedProfile.registration -%define CIMV2_REG %{_datadir}/%{name}/{HostedResourcePool,ElementCapabilities,HostedService,HostedDependency,ElementConformsToProfile,HostedAccessPoint}.registration -%define CIMV2_MOF %{_datadir}/%{name}/{HostedResourcePool,ElementCapabilities,HostedService,HostedDependency,RegisteredProfile,ComputerSystem,ElementConformsToProfile,HostedAccessPoint}.mof +%define INTEROP_MOF %{_datadir}/%{name}/ComputerSystem.mof \\\ + %{_datadir}/%{name}/HostSystem.mof \\\ + %{_datadir}/%{name}/RegisteredProfile.mof \\\ + %{_datadir}/%{name}/DiskPool.mof \\\ + %{_datadir}/%{name}/MemoryPool.mof \\\ + %{_datadir}/%{name}/NetPool.mof \\\ + %{_datadir}/%{name}/ProcessorPool.mof \\\ + %{_datadir}/%{name}/VSMigrationService.mof \\\ + %{_datadir}/%{name}/ElementConformsToProfile.mof \\\ + %{_datadir}/%{name}/ReferencedProfile.mof \\\ + %{_datadir}/%{name}/AllocationCapabilities.mof + +%define PGINTEROP_REG %{_datadir}/%{name}/RegisteredProfile.registration \\\ + %{_datadir}/%{name}/ElementConformsToProfile.registration \\\ + %{_datadir}/%{name}/ReferencedProfile.registration + +%define PGINTEROP_MOF %{_datadir}/%{name}/RegisteredProfile.mof \\\ + %{_datadir}/%{name}/ElementConformsToProfile.mof \\\ + %{_datadir}/%{name}/ReferencedProfile.mof + +%define CIMV2_REG %{_datadir}/%{name}/HostedResourcePool.registration \\\ + %{_datadir}/%{name}/ElementCapabilities.registration \\\ + %{_datadir}/%{name}/HostedService.registration \\\ + %{_datadir}/%{name}/HostedDependency.registration \\\ + %{_datadir}/%{name}/ElementConformsToProfile.registration \\\ + %{_datadir}/%{name}/HostedAccessPoint.registration + +%define CIMV2_MOF %{_datadir}/%{name}/HostedResourcePool.mof \\\ + %{_datadir}/%{name}/ElementCapabilities.mof \\\ + %{_datadir}/%{name}/HostedService.mof \\\ + %{_datadir}/%{name}/HostedDependency.mof \\\ + %{_datadir}/%{name}/RegisteredProfile.mof \\\ + %{_datadir}/%{name}/ComputerSystem.mof \\\ + %{_datadir}/%{name}/ElementConformsToProfile.mof \\\ + %{_datadir}/%{name}/HostedAccessPoint.mof %pre # _If_ there is already a version of this installed, we must deregister @@ -118,12 +265,12 @@ then fi %if 0%{?fedora} >= 17 || 0%{?rhel} >= 7 - if [ "`systemctl is-active tog-pegasus.service`" = "active" ] + if [ "`systemctl is-active tog-pegasus.service 2> /dev/null`" = "active" ] then systemctl restart tog-pegasus.service fi - if [ "`systemctl is-active sblim-sfcb.service`" = "active" ] + if [ "`systemctl is-active sblim-sfcb.service 2> /dev/null`" = "active" ] then systemctl restart sblim-sfcb.service fi -- 1.7.9.5 From jferlan at redhat.com Mon Nov 11 20:58:30 2013 From: jferlan at redhat.com (John Ferlan) Date: Mon, 11 Nov 2013 15:58:30 -0500 Subject: [Libvirt-cim] [RESEND PATCH 0/3] libvirt-cim: Fix provider registration In-Reply-To: <1383642213-10260-1-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1383642213-10260-1-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <528144F6.6030907@redhat.com> On 11/05/2013 04:03 AM, Viktor Mihajlovski wrote: > The following series fixes two issues in the provider registration > process of the libvirt-cim RPM package. > First, the RPM scriptlets didn't properly reflect the invocation > sequence during an RPM upgrade. This is fixed by the first patch. > Further, the registration using wildcards (*.mof) failed because of > an inter-MOF depedency. Fixed by the second patch. > Finally, the class deletion sequence in provider-register.sh was > prone to failure, resulting in residual entries in the Pegasus > class repository. Addressed by third patch. > > Note: since the redhat.com mailing lists are bouncing my emails > I am sending from a different account temporarily until this is > resolved. > > Viktor Mihajlovski (3): > build: Fix incorrect provider registration in upgrade path > build: Fix provider registration issues > schema: Fix class removal with Pegasus > > libvirt-cim.spec.in | 282 ++++++++++++++++++++++++++++++++++++++++---------- > provider-register.sh | 8 +- > 2 files changed, 231 insertions(+), 59 deletions(-) > I'm much of an expert in these matters - in fact pre-novice would summarize my experience with RPM files :-) Things seem reasonable; however, I'm in a bit of a quandary now as I cannot get my libvirt-cim environment to work in order to test. This past Friday I did do a yum update and since that time I cannot seem to get the libvirt-cim provider and cimserver to talk - quite frustrating. I was actually hoping to spend this week reviewing and testing libvirt-cim patches... Now I'm just trying to figure out why two best friends won't speak to each other any more :-) John From cngesaint at gmail.com Tue Nov 12 03:04:38 2013 From: cngesaint at gmail.com (Xu Wang) Date: Tue, 12 Nov 2013 11:04:38 +0800 Subject: [Libvirt-cim] Fwd: [PATCH V2 00/48] Solution to solve unsupported tag missing issue In-Reply-To: <1382928377-16331-1-git-send-email-gesaint@linux.vnet.ibm.com> References: <1382928377-16331-1-git-send-email-gesaint@linux.vnet.ibm.com> Message-ID: <52819AC6.8000609@gmail.com> Dear all, There are more and more bugs were reported as unsupported tags missing during resource update. And this issue is becoming more and more serious. My mission is solving this category bugs from designing by the end of this year. So could you help me review these patches and give me your suggestion? I hope I can get more feedback from community and solve it as soon as possible. If there is anything I can explain please reply me. Thank you very much. Sincerely yours, Xu Wang -------- ???? -------- ??: [PATCH V2 00/48] Solution to solve unsupported tag missing issue ??: Mon, 28 Oct 2013 10:45:29 +0800 ???: Xu Wang ???: libvirt-cim at redhat.com ??: jferlan at redhat.com, mihajlov at linux.vnet.ibm.com, Xu Wang What's new in V2: 1. Rebase commit to the current upstream. 2. Update console device support. 3. Add id and parent_id to handle there are more than two sub-node but with the same issue. 4. Add status (ACTIVE/INACTIVE) to identify if a node was fetched. 5. Adjust the original designing plan. Now after fetched a node from link list just change its status from ACTIVE to INACTIVE instead of delete it. Purpose of this update is to make link list keep the structure of xml and regenerate xml easily. 6. Necessary comments added. 7. Add function (compare_param_int/compare_param_str) to adjust the result of comparation (if any parameter is NULL/-1 the comparation result is TRUE instead of FALSE. 8. Bugs fixing. Such as attribute 'arch' under of , of parsing, etc. -------------------------------------------------------------------------- These patches are based on commit 04bfeb825e5e155ffa72119253de608ccf3bd72b. So I may need more work on rebasing. Most of bugs reported recently are about some tags libvirt-cim doesn't support will be dropped after resource updated. So a new member was added into every virt_device and domain structure. And a new structure named unknown device was added to save those tags new added. The original implementaion is reading every member of virt_device structure from xml and save it. But the defect of this is some tags were not a member of virt_device were dropped. After resource updated, every data in the virt_device will be used to regenerate new xml. Hence the tags unsupported above, disappeared. So I added a member into every virt_device and domain structure, 'others'. It's a link list and used to save all data read from xml. Another new structure 'unknown_device' was added to save data except libvirt-cim could recognize. The new implementation is, firstly parse_*_device() could read all nodes and properties from xml and save them into 'others' link list. Then every member of virt_device will fetch data from others link list and save it. So that nodes in the 'others' link list could be saved until they are used to re- generate xml. After resource updating finished, libvirt-cim will call *_xml() to generate xml. The new process of generating xml like this, firstly all data in the members of virt_device will be restored into 'others' link list, then a function (others_to_xml) will use this link list to generate xml. Some points I have updated, 1. 'others' link list has to be processed in _get_proc_device() and _get_mem_device(). They should be copied into new data structure. 2. If resource updating happened, others field should be cleared because this device has been changed and they are useless. Besides above some logic may be a little strange or boring. Implemention like that is just to be compatible with upper layer functions (to make changes as less as possible). After keep a balance I decided to devide the whole xml into several parts: ------------------------others in domain to save unsupported sub-nodes of domain -some fields (devices, mem, vcpu...) will be skiped because -they have their own parsing functions. xxx xxx xxx xxx xxx------- others in virt_device to save unsupported tags of this device xxx ...----------- unknown_device to save unsupported device except like ,,etc. Hence, all nodes read from xml will be restored after xml generation. Xu Wang (48): Add others member for saving unsupported tag and unknown device Add others and unknown_device clean up Add basic operations for reading data from xml node Fix xml parsing algorithm for parse_fs_device() Fix xml parsing algorithm for parse_block_device() Fix xml parsing algorithm for parse_vsi_device() Fix xml parsing algorithm for parse_net_device() Fix xml parsing algorithm for parse_vcpu_device() Fix xml parsing algorithm for parse_emu_device() Fix xml parsing algorithm for parse_mem_device() Fix xml parsing algorithm for parse_console_device() Fix xml parsing algorithm for parse_graphics_device() Fix xml parsing algorithm for parse_input_device() Add parse_unknown_device() Add parse_devices() for unknown type in get_dominfo_from_xml() Fix xml parsing algorithm in parse_domain() Fix xml parsing algorithm in parse_os() Fix xml parsing algorithm in parse_feature() Add dup function for device copy Add type CIM_RES_TYPE_DELETED and modify type as it after resource_del Add basic functions about converting others link list to xml Fix xml generation algorithm in console_xml() Fix xml generation algorithm in disk_block_xml() Fix xml generation algorithm in disk_file_xml() Fix xml generation algorithm in disk_fs_xml() Fix xml generation algorithm in set_net_vsi() Fix xml generation algorithm in set_net_source() Fix xml generation algorithm in bridge_net_to_xml() Fix xml generation algorithm in net_xml() Fix xml generation algorithm in vcpu_xml() Fix xml generation algorithm in cputune_xml() Fix xml generation algorithm in mem_xml() Fix xml generation algorithm in emu_xml() Fix xml generation algorithm in graphics_vnc_xml() Fix xml generation algorithm in input_xml() Fix xml generation algorithm in system_xml() Fix xml generation algorithm in _xenpv_os_xml() Fix xml generation algorithm in _fv_bootlist_xml() Fix xml generation algorithm in _xenfv_os_xml() Fix xml generation algorithm in _kvm_os_xml() Fix xml generation algorithm in _lxc_os_xml() Fix xml generation algorithm in os_xml() Fix xml generation algorithm in features_xml() Add functions for xml generation of unknown devices Fix xml generation algorithm in system_to_xml() Add cleanup_others() calling during rasd_to_vdev() Add others handling in _get_mem_device() Add dup_others() into _get_proc_device() libxkutil/device_parsing.c | 1959 ++++++++++++++++++++++----- libxkutil/device_parsing.h | 64 + libxkutil/xmlgen.c | 2103 +++++++++++++++++++++++------ src/Virt_VirtualSystemManagementService.c | 20 +- src/svpc_types.h | 1 + 5 files changed, 3445 insertions(+), 702 deletions(-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From jferlan at redhat.com Tue Nov 12 12:05:48 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 12 Nov 2013 07:05:48 -0500 Subject: [Libvirt-cim] Fwd: [PATCH V2 00/48] Solution to solve unsupported tag missing issue In-Reply-To: <52819AC6.8000609@gmail.com> References: <1382928377-16331-1-git-send-email-gesaint@linux.vnet.ibm.com> <52819AC6.8000609@gmail.com> Message-ID: <5282199C.2000205@redhat.com> On 11/11/2013 10:04 PM, Xu Wang wrote: > Dear all, > There are more and more bugs were reported as unsupported tags missing > during resource > update. And this issue is becoming more and more serious. My mission is > solving this category > bugs from designing by the end of this year. So could you help me review > these patches and > give me your suggestion? I hope I can get more feedback from community > and solve it as soon > as possible. If there is anything I can explain please reply me. Thank > you very much. > > Sincerely yours, > Xu Wang I was reviewing this at the end of last month and was the last "good" run I got out of my cimtest before a 10/30/13 update to tog-pegasus seems to have removed the ability to use libvirt-cim as a provider. I have a feeling the issue has something to do with the PG_InterOp definitions, but I'm not quite sure as the wording on the OpenPegasus pages regarding this isn't clear to me. Unfortunately it seems the community has been reduced in scope to a very few and while libvirt continues to move forward libvirt-cim languishes behind. Lately finding time to review 50+ patches has been a challenge. In any case, while I haven't had the time to get into the details of the changes it seems the issue is that you have devised a way to ensure XML changes to libvirt aren't "lost" by libvirt-cim. I have to try and figure out another way to review/test the patches since my current means of my f19 laptop doesn't with tog-pegasus 2.12.1-8 no longer works. John From daniel.hansel at linux.vnet.ibm.com Tue Nov 12 16:28:42 2013 From: daniel.hansel at linux.vnet.ibm.com (Daniel Hansel) Date: Tue, 12 Nov 2013 17:28:42 +0100 Subject: [Libvirt-cim] [RESEND PATCH] libvirt-cim: Changed resource type value EMU Message-ID: <1384273723-12730-1-git-send-email-daniel.hansel@linux.vnet.ibm.com> To avoid conflicts with CIM resource type OTHER the value of the resource type EMU is changed. Signed-off-by: Daniel Hansel --- src/svpc_types.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/svpc_types.h b/src/svpc_types.h index 66f65e4..4600196 100644 --- a/src/svpc_types.h +++ b/src/svpc_types.h @@ -30,12 +30,12 @@ #define CIM_RES_TYPE_MEM 4 #define CIM_RES_TYPE_NET 10 #define CIM_RES_TYPE_DISK 17 -#define CIM_RES_TYPE_EMU 1 #define CIM_RES_TYPE_GRAPHICS 24 #define CIM_RES_TYPE_INPUT 13 #define CIM_RES_TYPE_UNKNOWN 1000 #define CIM_RES_TYPE_IMAGE 32768 #define CIM_RES_TYPE_CONSOLE 32769 +#define CIM_RES_TYPE_EMU 32770 #define CIM_RES_TYPE_COUNT 7 const static int cim_res_types[CIM_RES_TYPE_COUNT] = -- 1.7.9.5 From daniel.hansel at de.ibm.com Tue Nov 12 16:43:54 2013 From: daniel.hansel at de.ibm.com (Daniel Hansel) Date: Tue, 12 Nov 2013 17:43:54 +0100 Subject: [Libvirt-cim] [RESEND PATCH 0/3] libvirt-cim: Fix provider registration In-Reply-To: <528144F6.6030907@redhat.com> References: <1383642213-10260-1-git-send-email-mihajlov@linux.vnet.ibm.com> <528144F6.6030907@redhat.com> Message-ID: <52825ACA.4000704@de.ibm.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 11.11.2013 21:58, John Ferlan wrote: > On 11/05/2013 04:03 AM, Viktor Mihajlovski wrote: >> The following series fixes two issues in the provider registration process of the libvirt-cim RPM package. First, the RPM scriptlets didn't properly reflect the invocation sequence during an >> RPM upgrade. This is fixed by the first patch. Further, the registration using wildcards (*.mof) failed because of an inter-MOF depedency. Fixed by the second patch. Finally, the class >> deletion sequence in provider-register.sh was prone to failure, resulting in residual entries in the Pegasus class repository. Addressed by third patch. >> >> Note: since the redhat.com mailing lists are bouncing my emails I am sending from a different account temporarily until this is resolved. >> >> Viktor Mihajlovski (3): build: Fix incorrect provider registration in upgrade path build: Fix provider registration issues schema: Fix class removal with Pegasus >> >> libvirt-cim.spec.in | 282 ++++++++++++++++++++++++++++++++++++++++---------- provider-register.sh | 8 +- 2 files changed, 231 insertions(+), 59 deletions(-) >> > > I'm much of an expert in these matters - in fact pre-novice would summarize my experience with RPM files :-) > > Things seem reasonable; however, I'm in a bit of a quandary now as I cannot get my libvirt-cim environment to work in order to test. This past Friday I did do a yum update and since that time I > cannot seem to get the libvirt-cim provider and cimserver to talk - quite frustrating. > > I was actually hoping to spend this week reviewing and testing libvirt-cim patches... Now I'm just trying to figure out why two best friends won't speak to each other any more :-) Hi John, could describe your libvirt-cim environment a bit more detailed (e.g. OS version, installed package versions of libvirt and libvirt-cim, cimserver, etc.)? Maybe we could figure out the error together. > > John > > _______________________________________________ Libvirt-cim mailing list Libvirt-cim at redhat.com https://www.redhat.com/mailman/listinfo/libvirt-cim > - -- Mit freundlichen Gr??en / Kind regards Daniel Hansel IBM Deutschland Research & Development GmbH Vorsitzende des Aufsichtsrats: Martina Koederitz Gesch?ftsf?hrung: Dirk Wittkopp Sitz der Gesellschaft: B?blingen Registergericht: Amtsgericht Stuttgart, HRB 243294 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSglrKAAoJEJszBMQcau4WQ3wP/1G1qsKhzOnPfW+joOcV+amd yDGrD+oKm7ivE64phe9YTfn69n5kyoPPLltgtGSSqNHHy6WAnBBi+r/HazSngWlQ K4QrPLPPyJIkHjtgVcNapx3hKSAhcAEqLvkUSn2SQAgv4YGpil3j31jh86fc52m8 meKx43834zASaDWjgEPBMJEM/aCqi++klmM1Hb7P5jwfdHkqwDmgyuIfBhwTAUfN 7BO66F8a1nEA4JUbpLy0tp9ajie9b8NxxPlzdUdx4KhWFytbhty/qG5Vr1sfkjYv HsKi+HG5HRJAnk8FdOLng7gwOTiOvReS7SSkgSwjznGX4PKJn2UzbXysK7Hm7mIx Siz8a9rw9WmF4IApTIf0IlPlQiAdBfa5TOglcjjRDyt4HioyAMNQdLqwTGNtcGJE cc0tDv7uQwMiAXmIqm4oEguNGi8iSQKIJlBjvJ7NcVe4d+3C/1Le/nTy6huPw/RS XPP6b2WeU8+7Irf1Pk+XfMMv0mXSJAPFWREUMcPETvK9VmQMI0I2hUaPo+50CHHp +p3uzcZEG9V3/XCD2JnxPhbYzAzbhYVTqXmE7HYDx9uk1bOZTxDNJeYRu8kAmieA HgkuyAhNrN7OkkSw5DJuD483bR48bDbrt7a7HLkPYwAm+/IAn+zRoDRpBQjdwdlx rK6VvRrYm6BvoEHIvwaH =sNd+ -----END PGP SIGNATURE----- From jferlan at redhat.com Tue Nov 12 17:34:04 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 12 Nov 2013 12:34:04 -0500 Subject: [Libvirt-cim] [RESEND PATCH 0/3] libvirt-cim: Fix provider registration In-Reply-To: <52825ACA.4000704@de.ibm.com> References: <1383642213-10260-1-git-send-email-mihajlov@linux.vnet.ibm.com> <528144F6.6030907@redhat.com> <52825ACA.4000704@de.ibm.com> Message-ID: <5282668C.8080000@redhat.com> >> >> I'm much of an expert in these matters - in fact pre-novice would summarize my experience with RPM files :-) >> >> Things seem reasonable; however, I'm in a bit of a quandary now as I cannot get my libvirt-cim environment to work in order to test. This past Friday I did do a yum update and since that time I >> cannot seem to get the libvirt-cim provider and cimserver to talk - quite frustrating. >> >> I was actually hoping to spend this week reviewing and testing libvirt-cim patches... Now I'm just trying to figure out why two best friends won't speak to each other any more :-) > Hi John, > > could describe your libvirt-cim environment a bit more detailed (e.g. OS version, installed package versions of libvirt and libvirt-cim, cimserver, etc.)? > > Maybe we could figure out the error together. > Sure - I am running f19 on a lenovo t530 - it's my work supplied laptop I generally use the top of the libvirt and libvirt-cim tree, although those didn't change when this issue was discovered. Prior to 10/30/13, I'm not sure which version of tog-pegasus* was installed; however, as of that day the following was installed: Oct 30 08:00:41 Updated: 2:tog-pegasus-libs-2.12.1-8.fc19.x86_64 Oct 30 08:01:08 Updated: 2:tog-pegasus-2.12.1-8.fc19.x86_64 Oct 30 08:01:39 Updated: 2:tog-pegasus-devel-2.12.1-8.fc19.x86_64 a 'cimprovider -l' does not return any KVM_ modules a sure sign of things not working... In order to help dig, I enabled debugging: cimconfig -s traceLevel=4 -c cimconfig -s traceComponents=ALL -c Looking at the cimserver.trc file I find these (sorry about the formatting my auto line wrap is on): 1384276573s-184895us: Repository [7944:139824043710528:FileBasedStore.cpp:631]: Namespace: root#PG_InterOp ignored -- subdirectories are not correctly formed 1384276573s-186825us: L10N [7944:139824043710528:MessageLoader.cpp:418]: Message ID = Common.InternalException.CANNOT_OPEN_DIRECTORY 1384276573s-187030us: Repository [7944:139824043710528:FileBasedStore.cpp:1347]: Namespace: root#PG_InterOp ignored -- subdirectories are not correctly formed 1384276573s-194463us: ProviderManager [7944:139824043710528:ProviderRegistrationManager.cpp:1934]: nameSpace = root/interop; className = PG_ProviderModule I think you get the picture though - there's something about PG_InterOp which is different. Since it's one of those things that 2.12.1-8 release notes discusses as changing in 2.12.1-4, I have a feeling whatever it is the libvirt-cim.spec does in order to "link up" as a provider, does not work in the same way any more. Doing a : wbemcli ein http://root:password at localhost:5988/root/virt:KVM_VirtualSystemManagementService gets: 1384277180s-587013us: ProviderManager [9934:140222156466240:ProviderRegistrationManager.cpp:395]: nameSpace = root/virt; className = KVM_VirtualSystemManagementService; className = root/virtkvm_virtualsystemmanagementserviceinstance 1384277180s-587020us: ProviderManager [9934:140222156466240:ProviderRegistrationManager.cpp:406]: Provider capability has not been registered yet. 1384277180s-587028us: Dispatcher [9934:140222156466240:CIMOperationRequestDispatcher.cpp:1465]: Provider for KVM_VirtualSystemManagementService not found. But no output from the wbemcli command. I did try "going backwards" and installing an older rpm, but was not successful. So I borrowed another RHEL6.5 system and am testing over there today. John From jferlan at redhat.com Tue Nov 12 21:13:21 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 12 Nov 2013 16:13:21 -0500 Subject: [Libvirt-cim] [PATCH 1/4] VSDC: Fix endianess issues In-Reply-To: <1381492055-11502-2-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1381492055-11502-1-git-send-email-mihajlov@linux.vnet.ibm.com> <1381492055-11502-2-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <528299F1.7000101@redhat.com> On 10/11/2013 07:47 AM, Viktor Mihajlovski wrote: > SetProperty was called with int instead of uint16_t in a few places. > This leads to failures on big-endian systems like s390. Found > by runnung cimstest. > > Signed-off-by: Viktor Mihajlovski > --- > src/Virt_SettingsDefineCapabilities.c | 16 ++++++++-------- > 1 file changed, 8 insertions(+), 8 deletions(-) > ACK John From jferlan at redhat.com Tue Nov 12 21:28:45 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 12 Nov 2013 16:28:45 -0500 Subject: [Libvirt-cim] [PATCH 2/4] FilterEntry: Fix endianness issues In-Reply-To: <1381492055-11502-3-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1381492055-11502-1-git-send-email-mihajlov@linux.vnet.ibm.com> <1381492055-11502-3-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <52829D8D.7040805@redhat.com> On 10/11/2013 07:47 AM, Viktor Mihajlovski wrote: > From: Thilo Boehm > > A number of CIM properties was set in an endianness-unsafe manner > leading to failures on big endian systems. > > Signed-off-by: Thilo Boehm > Signed-off-by: Viktor Mihajlovski > --- > src/Virt_FilterEntry.c | 46 ++++++++++++++++++++++++---------------------- > 1 file changed, 24 insertions(+), 22 deletions(-) > > diff --git a/src/Virt_FilterEntry.c b/src/Virt_FilterEntry.c > index b7042da..e41b4b6 100644 > --- a/src/Virt_FilterEntry.c > +++ b/src/Virt_FilterEntry.c > @@ -59,8 +59,8 @@ struct rule_data_t { > const char *dstportend; > }; > > -static int octets_from_mac(const char * s, unsigned int *buffer, > - unsigned int size) > +static int octets_from_mac(const char * s, uint8_t *buffer, > + unsigned int size) > { > unsigned int _buffer[6]; > unsigned int i, n = 0; > @@ -86,8 +86,8 @@ static int octets_from_mac(const char * s, unsigned int *buffer, > return n; > } > > -static int octets_from_ip(const char * s, unsigned int *buffer, > - unsigned int size) > +static int octets_from_ip(const char * s, uint8_t *buffer, > + unsigned int size) > { > struct in6_addr addr; > unsigned int family = 0; > @@ -116,7 +116,8 @@ static int octets_from_ip(const char * s, unsigned int *buffer, > return n; > } > > -static CMPIArray *octets_to_cmpi(const CMPIBroker *broker, unsigned int *bytes, int size) > +static CMPIArray *octets_to_cmpi(const CMPIBroker *broker, uint8_t *bytes, > + int size) > { > CMPIStatus s = {CMPI_RC_OK, NULL}; > CMPIArray *array = NULL; > @@ -216,7 +217,7 @@ static int convert_action(const char *s) > return action; > } > > -static unsigned long convert_protocol_id(const char *s) > +static uint16_t convert_protocol_id(const char *s) > { > enum {NONE = 0, IPV4 = 2048, ARP = 2054, RARP = 32821, IPV6 = 34525} id = NONE; > > @@ -239,7 +240,7 @@ static void convert_mac_rule_to_instance( > CMPIInstance *inst, > const CMPIBroker *broker) > { > - unsigned int bytes[48]; > + uint8_t bytes[48]; > unsigned int size = 0; > CMPIArray *array = NULL; > > @@ -280,7 +281,7 @@ static void convert_mac_rule_to_instance( > (CMPIValue *)&array, CMPI_uint8A); > > if (rule->var.mac.protocol_id != NULL) { > - unsigned long n = convert_protocol_id(rule->var.mac.protocol_id); > + uint16_t n = convert_protocol_id(rule->var.mac.protocol_id); > > /* Unknown protocolid string. Try converting from hexadecimal value */ > if (n == 0) > @@ -366,18 +367,19 @@ static void convert_ip_rule_to_instance( > CMPIInstance *inst, > const CMPIBroker *broker) > { > - unsigned int bytes[48]; > + uint8_t bytes[48]; > unsigned int size = 0; > - unsigned int n = 0; > + uint8_t ipver_num = 0; > + uint16_t port_num = 0; > CMPIArray *array = NULL; > struct rule_data_t rule_data; > > if (strstr(rule->protocol_id, "v6")) > - n = 6; > + ipver_num = 6; > else > - n = 4; > + ipver_num = 4; > > - CMSetProperty(inst, "HdrIPVersion",(CMPIValue *)&n, CMPI_uint8); > + CMSetProperty(inst, "HdrIPVersion",(CMPIValue *)&ipver_num, CMPI_uint8); > > fill_rule_data(rule, &rule_data); > > @@ -480,27 +482,27 @@ static void convert_ip_rule_to_instance( > } > > if (rule_data.srcportstart) { > - n = atoi(rule_data.srcportstart); > + port_num = atoi(rule_data.srcportstart); Hmm.. technically atoi() returns an 'int'... In libvirt code this would be a call to a function which does a strtoul() and makes sure there's no overflow, etc. However, since this code is full of other situations such as this, I'm "OK" with the way it's done... > CMSetProperty(inst, "HdrSrcPortStart", > - (CMPIValue *)&n, CMPI_uint16); > + (CMPIValue *)&port_num, CMPI_uint16); > } > > if (rule_data.srcportend) { > - n = atoi(rule_data.srcportend); > + port_num = atoi(rule_data.srcportend); > CMSetProperty(inst, "HdrSrcPortEnd", > - (CMPIValue *)&n, CMPI_uint16); > + (CMPIValue *)&port_num, CMPI_uint16); > } > > if (rule_data.dstportstart) { > - n = atoi(rule_data.dstportstart); > + port_num = atoi(rule_data.dstportstart); > CMSetProperty(inst, "HdrDestPortStart", > - (CMPIValue *)&n, CMPI_uint16); > + (CMPIValue *)&port_num, CMPI_uint16); > } > > if (rule_data.dstportend) { > - n = atoi(rule_data.dstportend); > + port_num = atoi(rule_data.dstportend); > CMSetProperty(inst, "HdrDestPortEnd", > - (CMPIValue *)&n, CMPI_uint16); > + (CMPIValue *)&port_num, CMPI_uint16); > } > } > > @@ -515,7 +517,7 @@ static CMPIInstance *convert_rule_to_instance( > const char *sys_name = NULL; > const char *sys_ccname = NULL; > const char *basename = NULL; > - int action, direction, priority = 0; > + uint16_t action, direction, priority = 0; This is declared an uint16_t here; however, later on it's stored as a signed int16: priority = convert_priority(rule->priority); CMSetProperty(inst, "Priority", (CMPIValue *)&priority, CMPI_sint16); Also in convert_filter_to_instance(), there's some similar oddities: ... int direction = 0, priority; ... CMSetProperty(inst, "Direction", (CMPIValue *)&direction, CMPI_uint16); priority = convert_priority(filter->priority); CMSetProperty(inst, "Priority", (CMPIValue *)&priority, CMPI_sint16); Where 'direction' is a signed value, but being stored unsigned and priority is treated as a signed value. Just let me know which way you'd like to have these adjusted, I'll make the change and then push the series. John > > void (*convert_f)(struct acl_rule*, CMPIInstance*, const CMPIBroker*); > > From jferlan at redhat.com Tue Nov 12 21:46:35 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 12 Nov 2013 16:46:35 -0500 Subject: [Libvirt-cim] [PATCH 3/4] VSSM: Fix endianness issue in domain properties In-Reply-To: <1381492055-11502-4-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1381492055-11502-1-git-send-email-mihajlov@linux.vnet.ibm.com> <1381492055-11502-4-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <5282A1BB.2030606@redhat.com> On 10/11/2013 07:47 AM, Viktor Mihajlovski wrote: > From: Thilo Boehm > > The properties for the on_xxx actions must be 16-bit values in > order to avoid failures on big endian systems. > > Signed-off-by: Thilo Boehm > Reviewed-by: Viktor Mihajlovski > --- > libxkutil/device_parsing.c | 2 +- > libxkutil/device_parsing.h | 8 ++++---- > src/Virt_VirtualSystemManagementService.c | 4 ++-- > 3 files changed, 7 insertions(+), 7 deletions(-) > ACK John From jferlan at redhat.com Tue Nov 12 22:16:19 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 12 Nov 2013 17:16:19 -0500 Subject: [Libvirt-cim] [PATCH 4/4] libxkutil: clean entire device structure to avoid memory corruption In-Reply-To: <1381492055-11502-5-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1381492055-11502-1-git-send-email-mihajlov@linux.vnet.ibm.com> <1381492055-11502-5-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <5282A8B3.10503@redhat.com> On 10/11/2013 07:47 AM, Viktor Mihajlovski wrote: > If cleanup_virt_device is called twice (e.g. during modify resource) > a double free can occur because only the dev substructure has > been memset to zero. Now zeroing the entire structure. > > Signed-off-by: Viktor Mihajlovski > --- > libxkutil/device_parsing.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > ACK John From jferlan at redhat.com Tue Nov 12 22:16:58 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 12 Nov 2013 17:16:58 -0500 Subject: [Libvirt-cim] [PATCH 1/5] RASD/schema: Add properties for device address representation In-Reply-To: <1381764584-18025-2-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1381764584-18025-1-git-send-email-mihajlov@linux.vnet.ibm.com> <1381764584-18025-2-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <5282A8DA.2000809@redhat.com> On 10/14/2013 11:29 AM, Viktor Mihajlovski wrote: > Certain device types are only identifiable uniquely via a device > address from the guest's perspective. Adding or removing devices > has the potential to change the address of other unrelated devices > and impact the guest's operation. Therefore it is less desirable > to rely on implicit device address allocation but rather to use > persistent device addresses. > > Depending on the device's bus type (PCI, SCSI, USB, CCW ...) the > address specification format can vary, see also the libvirt > domain XML documentation. To account for the various formats > device addresses are specified with two array properties in the > respective RASD classes: AddressProperties and AddressValues. > The former contains a list of address property names and the latter > the values. > > E.g., a PCI address is specified by the properties domain, bus, > slot and function. Therefore, for a PCI device RASD we could > have: > > AddressProperties = ['type', 'domain','bus', 'slot', 'function'] > AddressValues = ['pci', '0x0000', '0x00', '0x01', '0x2'] > > resulting in a libvirt address element: > >
function='0x2'/> > > Initially, we support only disk and network devices for KVM guests. > > Signed-off-by: Viktor Mihajlovski > --- > schema/ResourceAllocationSettingData.mof | 12 ++++++++++++ > 1 file changed, 12 insertions(+) > ACK John From jferlan at redhat.com Tue Nov 12 22:34:09 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 12 Nov 2013 17:34:09 -0500 Subject: [Libvirt-cim] [PATCH 2/5] libxkutil: Support for device addresses In-Reply-To: <1381764584-18025-3-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1381764584-18025-1-git-send-email-mihajlov@linux.vnet.ibm.com> <1381764584-18025-3-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <5282ACE1.1080607@redhat.com> On 10/14/2013 11:29 AM, Viktor Mihajlovski wrote: > New data type for device addresses based on the generic > "address" element of the libvirt domain XML. > Contains XML parsing and generation code for device addresses. > > Signed-off-by: Viktor Mihajlovski > --- > libxkutil/device_parsing.c | 113 ++++++++++++++++++++++++++++++++++++++++++++ > libxkutil/device_parsing.h | 11 +++++ > libxkutil/xmlgen.c | 28 +++++++++++ > 3 files changed, 152 insertions(+) > > diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c > index 076bec0..56e39c7 100644 > --- a/libxkutil/device_parsing.c > +++ b/libxkutil/device_parsing.c > @@ -63,6 +63,22 @@ > /* Device parse function */ > typedef int (*dev_parse_func_t)(xmlNode *, struct virt_device **); > > +static void cleanup_device_address(struct device_address *addr) > +{ > + int i; > + if (addr == NULL) > + return; > + > + for (i = 0; i < addr->ct; i++) { > + free(addr->key[i]); > + free(addr->value[i]); > + } > + > + free(addr->key); > + free(addr->value); > + addr->ct = 0; > +} > + > static void cleanup_disk_device(struct disk_device *dev) > { > if (dev == NULL) > @@ -77,6 +93,7 @@ static void cleanup_disk_device(struct disk_device *dev) > free(dev->virtual_dev); > free(dev->bus_type); > free(dev->access_mode); > + cleanup_device_address(&dev->address); > } > > static void cleanup_vsi_device(struct vsi_device *dev) > @@ -107,6 +124,7 @@ static void cleanup_net_device(struct net_device *dev) > free(dev->filter_ref); > free(dev->poolid); > cleanup_vsi_device(&dev->vsi); > + cleanup_device_address(&dev->address); > } > > static void cleanup_emu_device(struct emu_device *dev) > @@ -351,6 +369,67 @@ char *get_node_content(xmlNode *node) > return buf; > } > > +int add_device_address_property(struct device_address *devaddr, > + const char *key, > + const char *value) > +{ > + char *k = NULL; > + char *v = NULL; > + char **list = NULL; > + > + if (key != NULL && value != NULL) { > + k = strdup(key); > + v = strdup(value); > + if (k == NULL || v == NULL) > + goto err; > + > + list = realloc(devaddr->key, sizeof(char*) * (devaddr->ct+1)); > + if (list == NULL) > + goto err; > + devaddr->key = list; > + > + list = realloc(devaddr->value, sizeof(char*) * (devaddr->ct+1)); > + if (list == NULL) There's a leak here since 'ct' isn't incremented until later, devaddr->key will have the 'list' from above and that'll either be lost or overwritten. I see two ways out... free(devaddr->key); devaddr->key = NULL; or let err: handle the free(list) below... list = devaddr->key; devaddr->key = NULL; Of course there's also list1 and list2... Let me know which you prefer, I can make the change before pushing... John > + goto err; > + devaddr->value = list; > + > + devaddr->key[devaddr->ct] = k; > + devaddr->value[devaddr->ct] = v; > + devaddr->ct += 1; > + return 1; > + } > + > + err: > + free(k); > + free(v); > + free(list); > + return 0; > +} > + > + > +static int parse_device_address(xmlNode *anode, struct device_address *devaddr) > +{ > + xmlAttr *attr = NULL; > + char *name = NULL; > + char *value = NULL; > + > + for (attr = anode->properties; attr != NULL; attr = attr->next) { > + name = (char*) attr->name; > + value = get_attr_value(anode, name); > + if (!add_device_address_property(devaddr, name, value)) > + goto err; > + free(value); > + } > + > + return 1; > + > + err: > + cleanup_device_address(devaddr); > + free(value); > + > + return 0; > +} > + > static int parse_fs_device(xmlNode *dnode, struct virt_device **vdevs) > { > struct virt_device *vdev = NULL; > @@ -386,6 +465,8 @@ static int parse_fs_device(xmlNode *dnode, struct virt_device **vdevs) > } > } else if (XSTREQ(child->name, "driver")) { > ddev->driver_type = get_attr_value(child, "type"); > + } else if (XSTREQ(child->name, "address")) { > + parse_device_address(child, &ddev->address); > } > } > > @@ -459,6 +540,8 @@ static int parse_block_device(xmlNode *dnode, struct virt_device **vdevs) > ddev->readonly = true; > } else if (XSTREQ(child->name, "shareable")) { > ddev->shareable = true; > + } else if (XSTREQ(child->name, "address")) { > + parse_device_address(child, &ddev->address); > } > } > > @@ -598,6 +681,8 @@ static int parse_net_device(xmlNode *inode, struct virt_device **vdevs) > ndev->filter_ref = get_attr_value(child, "filter"); > } else if (XSTREQ(child->name, "virtualport")) { > parse_vsi_device(child, ndev); > + } else if (XSTREQ(child->name, "address")) { > + parse_device_address(child, &ndev->address); > #if LIBVIR_VERSION_NUMBER >= 9000 > } else if (XSTREQ(child->name, "bandwidth")) { > /* Network QoS bandwidth support */ > @@ -1167,6 +1252,32 @@ static int parse_devices(const char *xml, struct virt_device **_list, int type) > return count; > } > > +static void duplicate_device_address(struct device_address *to, const struct device_address *from) > +{ > + int i; > + > + if (from == NULL || to == NULL || from->ct == 0) > + return; > + > + to->ct = from->ct; > + to->key = calloc(from->ct, sizeof(char*)); > + to->value = calloc(from->ct, sizeof(char*)); > + if (to->key == NULL || to->value == NULL) > + goto err; > + > + for (i = 0; i < from->ct; i++) { > + to->key[i] = strdup(from->key[i]); > + to->value[i] = strdup(from->value[i]); > + if (to->key[i] == NULL || to->value[i] == NULL) > + goto err; > + } > + > + return; > + > + err: > + cleanup_device_address(to); > +} > + > struct virt_device *virt_device_dup(struct virt_device *_dev) > { > struct virt_device *dev; > @@ -1196,6 +1307,7 @@ struct virt_device *virt_device_dup(struct virt_device *_dev) > DUP_FIELD(dev, _dev, dev.net.vsi.profile_id); > dev->dev.net.reservation = _dev->dev.net.reservation; > dev->dev.net.limit = _dev->dev.net.limit; > + duplicate_device_address(&dev->dev.net.address, &_dev->dev.net.address); > } else if (dev->type == CIM_RES_TYPE_DISK) { > DUP_FIELD(dev, _dev, dev.disk.type); > DUP_FIELD(dev, _dev, dev.disk.device); > @@ -1209,6 +1321,7 @@ struct virt_device *virt_device_dup(struct virt_device *_dev) > dev->dev.disk.disk_type = _dev->dev.disk.disk_type; > dev->dev.disk.readonly = _dev->dev.disk.readonly; > dev->dev.disk.shareable = _dev->dev.disk.shareable; > + duplicate_device_address(&dev->dev.disk.address, &_dev->dev.disk.address); > } else if (dev->type == CIM_RES_TYPE_MEM) { > dev->dev.mem.size = _dev->dev.mem.size; > dev->dev.mem.maxsize = _dev->dev.mem.maxsize; > diff --git a/libxkutil/device_parsing.h b/libxkutil/device_parsing.h > index 4beac5c..92427c1 100644 > --- a/libxkutil/device_parsing.h > +++ b/libxkutil/device_parsing.h > @@ -33,6 +33,12 @@ > > #include "../src/svpc_types.h" > > +struct device_address { > + uint32_t ct; > + char **key; > + char **value; > +}; > + > struct vsi_device { > char *vsi_type; > char *manager_id; > @@ -56,6 +62,7 @@ struct disk_device { > char *bus_type; > char *cache; > char *access_mode; /* access modes for DISK_FS (filesystem) type */ > + struct device_address address; > }; > > struct net_device { > @@ -70,6 +77,7 @@ struct net_device { > uint64_t reservation; > uint64_t limit; > struct vsi_device vsi; > + struct device_address address; > }; > > struct mem_device { > @@ -257,6 +265,9 @@ int get_devices(virDomainPtr dom, struct virt_device **list, int type, > void cleanup_virt_device(struct virt_device *dev); > void cleanup_virt_devices(struct virt_device **devs, int count); > > +int add_device_address_property(struct device_address *devaddr, > + const char *key, const char *value); > + > char *get_node_content(xmlNode *node); > char *get_attr_value(xmlNode *node, char *attrname); > > diff --git a/libxkutil/xmlgen.c b/libxkutil/xmlgen.c > index 7e8801d..40e2905 100644 > --- a/libxkutil/xmlgen.c > +++ b/libxkutil/xmlgen.c > @@ -178,6 +178,26 @@ static const char *console_xml(xmlNodePtr root, struct domain *dominfo) > BAD_CAST cdev->target_type); > } > } > + > + return NULL; > +} > + > +static char *device_address_xml(xmlNodePtr root, struct device_address *addr) > +{ > + int i; > + xmlNodePtr address; > + > + if (addr == NULL || addr->ct == 0) > + return NULL; > + > + address = xmlNewChild(root, NULL, BAD_CAST "address", NULL); > + if (address == NULL) > + return XML_ERROR; > + > + for (i = 0; i < addr->ct; i++) { > + xmlNewProp(address, BAD_CAST addr->key[i] , BAD_CAST addr->value[i]); > + } > + > return NULL; > } > > @@ -225,6 +245,9 @@ static char *disk_block_xml(xmlNodePtr root, struct disk_device *dev) > if (dev->shareable) > xmlNewChild(disk, NULL, BAD_CAST "shareable", NULL); > > + if (dev->address.ct > 0) > + return device_address_xml(disk, &dev->address); > + > return NULL; > } > > @@ -279,6 +302,8 @@ static const char *disk_file_xml(xmlNodePtr root, struct disk_device *dev) > if (dev->shareable) > xmlNewChild(disk, NULL, BAD_CAST "shareable", NULL); > > + if (dev->address.ct > 0) > + return device_address_xml(disk, &dev->address); > > return NULL; > } > @@ -520,6 +545,9 @@ static const char *net_xml(xmlNodePtr root, struct domain *dominfo) > } > #endif > > + if (dev->dev.net.address.ct > 0) > + msg = device_address_xml(nic, &dev->dev.net.address); > + > if (STREQ(dev->dev.net.type, "network")) { > msg = set_net_source(nic, net, "network"); > } else if (STREQ(dev->dev.net.type, "bridge")) { > From jferlan at redhat.com Tue Nov 12 22:34:40 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 12 Nov 2013 17:34:40 -0500 Subject: [Libvirt-cim] [PATCH 3/5] RASD: Support for device address properties In-Reply-To: <1381764584-18025-4-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1381764584-18025-1-git-send-email-mihajlov@linux.vnet.ibm.com> <1381764584-18025-4-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <5282AD00.1010303@redhat.com> On 10/14/2013 11:29 AM, Viktor Mihajlovski wrote: > This change allows to enumerate KVM disk and network RASDs containing > device addresses. A new function set_rasd_device_address fills the > CIM instance properties from a device_address structure. > > Signed-off-by: Viktor Mihajlovski > --- > src/Virt_RASD.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 73 insertions(+) > ACK John From jferlan at redhat.com Tue Nov 12 22:42:21 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 12 Nov 2013 17:42:21 -0500 Subject: [Libvirt-cim] [PATCH 4/5] VSMS: Support for device addresses In-Reply-To: <1381764584-18025-5-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1381764584-18025-1-git-send-email-mihajlov@linux.vnet.ibm.com> <1381764584-18025-5-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <5282AECD.2040301@redhat.com> On 10/14/2013 11:29 AM, Viktor Mihajlovski wrote: > This allows to define KVM guests with persistent device addresses > for disks and network interfaces. > The new function rasd_to_device_address extracts the address > properties from the disk or network RASD and fills the > device_address sub-structure of the affected virt_device. > > Signed-off-by: Viktor Mihajlovski > --- > src/Virt_VirtualSystemManagementService.c | 51 ++++++++++++++++++++++++++++- > 1 file changed, 50 insertions(+), 1 deletion(-) > ACK John From jferlan at redhat.com Tue Nov 12 22:48:17 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 12 Nov 2013 17:48:17 -0500 Subject: [Libvirt-cim] [PATCH 5/5] VSMS: Improve device cleanup In-Reply-To: <1381764584-18025-6-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1381764584-18025-1-git-send-email-mihajlov@linux.vnet.ibm.com> <1381764584-18025-6-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <5282B031.8030202@redhat.com> On 10/14/2013 11:29 AM, Viktor Mihajlovski wrote: > Use cleanup_virt_device instead of single free's all over > the place in net_rasd_to_vdev and disk_rasd_to_vdev. > Further, make sure that the device type is always set > independent of the implementation of the xxx_rasd_to_vdev > functions. > > Signed-off-by: Viktor Mihajlovski > --- > src/Virt_VirtualSystemManagementService.c | 35 +++++++---------------------- > 1 file changed, 8 insertions(+), 27 deletions(-) > ACK John From jferlan at redhat.com Tue Nov 12 23:29:38 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 12 Nov 2013 18:29:38 -0500 Subject: [Libvirt-cim] [PATCH] VSMS: fv_vssd_to_domain() resolve Coverity error Message-ID: <1384298978-31672-1-git-send-email-jferlan@redhat.com> Coverity discovered that the free(domain->os_info.fv.arch) and then usage later on during get_default_machine() and get_default_emulator() calls could result in using free()'d memory. If the 'cu_get_str_prop() failed or capsinfo == NULL, then the fv.arch wouldn't necessarily be strdup()'d. Passing a NULL os_info.fv_arch into the get*() API's is fine since they'll call findDomainInfo() which can handle a NULL arch value. Also added an initialization of val just to be safe. I don't think it's necessary though. --- NOTE: I found this during a Coverity run applying the endianness patches. For some reason Coverity "woke up" and saw this even though it hasn't found this issue in a couple months of runs since the changes to this module were made. See commit id '117dabb9'. src/Virt_VirtualSystemManagementService.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/Virt_VirtualSystemManagementService.c b/src/Virt_VirtualSystemManagementService.c index d51f230..9f8b5b9 100644 --- a/src/Virt_VirtualSystemManagementService.c +++ b/src/Virt_VirtualSystemManagementService.c @@ -464,7 +464,7 @@ static int fv_vssd_to_domain(CMPIInstance *inst, { int ret = 1; int retr; - const char *val; + const char *val = NULL; const char *domtype = NULL; const char *ostype = "hvm"; struct capabilities *capsinfo = NULL; @@ -494,6 +494,7 @@ static int fv_vssd_to_domain(CMPIInstance *inst, } free(domain->os_info.fv.arch); + domain->os_info.fv.arch = NULL; retr = cu_get_str_prop(inst, "Arch", &val); if (retr != CMPI_RC_OK) { if (capsinfo != NULL) { /* set default */ -- 1.8.3.1 From jferlan at redhat.com Wed Nov 13 00:21:54 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 12 Nov 2013 19:21:54 -0500 Subject: [Libvirt-cim] [PATCH 0/2] cimtest support for consoles In-Reply-To: <1381489788-21481-1-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1381489788-21481-1-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <5282C622.9060501@redhat.com> On 10/11/2013 07:09 AM, Viktor Mihajlovski wrote: > This series extends the KVM cimtests with the verification of > the console instrumentation. > If the tested libvirt-cim provider has the necessary release > level, the test domains are instantiated with a console. > The other patch deals with the fact that certain device types > (like graphics) are supported on s390. > > I ran the tests on an x86 system as well seeing no regressions, > but an independent test run would be welcome... > > Viktor Mihajlovski (2): > cimtest: Adding support for ConsoleRASD > cimtest: Fix s390 failures > Not sure how to approach this series. On the one hand things look good; however, when run with Xu Wang's 48 patch series it seems there's assumptions in one set of patches or the other. In particular, there's numerous failures with the libvirt message "Only the first console can be a serial port", which means for some reason there's more than one console in the guest configured to use the serial line. Since patch 1/2 deals with console's I have to assume that it's the reason/cause for what I've seen. I haven't had the time to dig deep on it though, but I guess I'd ask - is the code that seems to add a console to a guest ever (or need to check) whether the guest already has one? Would the 48 patch series somehow "assume" a need to put a console in an others list. Just throwing random ideas out there. It's been a long day. John From jferlan at redhat.com Wed Nov 13 00:44:41 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 12 Nov 2013 19:44:41 -0500 Subject: [Libvirt-cim] [RESEND PATCH] libvirt-cim: Changed resource type value EMU In-Reply-To: <1384273723-12730-1-git-send-email-daniel.hansel@linux.vnet.ibm.com> References: <1384273723-12730-1-git-send-email-daniel.hansel@linux.vnet.ibm.com> Message-ID: <5282CB79.1060804@redhat.com> On 11/12/2013 11:28 AM, Daniel Hansel wrote: > To avoid conflicts with CIM resource type OTHER the value of the > resource type EMU is changed. > > Signed-off-by: Daniel Hansel > --- > src/svpc_types.h | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > ACK John From jferlan at redhat.com Wed Nov 13 00:59:11 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 12 Nov 2013 19:59:11 -0500 Subject: [Libvirt-cim] [PATCH V2 01/48] Add others member for saving unsupported tag and unknown device In-Reply-To: <1382928377-16331-2-git-send-email-gesaint@linux.vnet.ibm.com> References: <1382928377-16331-1-git-send-email-gesaint@linux.vnet.ibm.com> <1382928377-16331-2-git-send-email-gesaint@linux.vnet.ibm.com> Message-ID: <5282CEDF.4090506@redhat.com> On 10/27/2013 10:45 PM, Xu Wang wrote: > Signed-off-by: Xu Wang > --- > libxkutil/device_parsing.h | 43 +++++++++++++++++++++++++++++++++++++++++++ > 1 files changed, 43 insertions(+), 0 deletions(-) > I am going to try to take these one at a time. Run them through cimtest singly to see if I can flesh out errors. I didn't get very far - 2/48 causes some failures in cimtest ... > diff --git a/libxkutil/device_parsing.h b/libxkutil/device_parsing.h > index 2803d6a..147551a 100644 > --- a/libxkutil/device_parsing.h > +++ b/libxkutil/device_parsing.h > @@ -33,6 +33,33 @@ > > #include "../src/svpc_types.h" > > +/* The structure for saving unknown tag in the xml */ > +enum others_type { > + TYPE_PROP, > + TYPE_NODE > +}; > + > +struct others { There's an extra space after the { which git am complains about > + /* To identify the different tags with same name */ > + int id; > + char *name; > + int parent_id; > + char *parent_name; > + enum others_type type; > + char *value; > + struct others *next; > + enum status { > + ACTIVE, > + INACTIVE > + }; Older compiler complains: ../libxkutil/device_parsing.h:54: warning: declaration does not declare anything New compiler where I don't ignore warnings stops: In file included from device_parsing.c:33:0: device_parsing.h:54:10: error: declaration does not declare anything [-Werror] }; ^ cc1: all warnings being treated as errors I'll have to see more to determine whether I like this pointer to pointer within structures... There's usually some assumption somewhere which causes a failure... John > +}; > + > +/* The structure for saving unknown device */ > +struct unknown_device { > + char *name; > + struct others *others; > +}; > + > struct vsi_device { > char *vsi_type; > char *manager_id; > @@ -56,6 +83,7 @@ struct disk_device { > char *bus_type; > char *cache; > char *access_mode; /* access modes for DISK_FS (filesystem) type */ > + struct others *others; > }; > > struct net_device { > @@ -70,6 +98,7 @@ struct net_device { > uint64_t reservation; > uint64_t limit; > struct vsi_device vsi; > + struct others *others; > }; > > struct mem_device { > @@ -78,16 +107,19 @@ struct mem_device { > enum { MEM_DUMP_CORE_NOT_SET, > MEM_DUMP_CORE_ON, > MEM_DUMP_CORE_OFF } dumpCore; > + struct others *others; > }; > > struct vcpu_device { > uint64_t quantity; > uint32_t weight; > uint64_t limit; > + struct others *others; > }; > > struct emu_device { > char *path; > + struct others *others; > }; > > struct vnc_device { > @@ -95,12 +127,14 @@ struct vnc_device { > char *host; > char *keymap; > char *passwd; > + struct others *others; > }; > > struct sdl_device { > char *display; > char *xauth; > char *fullscreen; > + struct others *others; > }; > > struct graphics_device { > @@ -109,6 +143,7 @@ struct graphics_device { > struct vnc_device vnc; > struct sdl_device sdl; > } dev; > + struct others *others; > }; > > struct path_device { > @@ -146,11 +181,13 @@ struct console_device { > struct udp_device udp; > } source_dev; > char *target_type; > + struct others *others; > }; > > struct input_device { > char *type; > char *bus; > + struct others *others; > }; > > struct virt_device { > @@ -164,6 +201,7 @@ struct virt_device { > struct graphics_device graphics; > struct console_device console; > struct input_device input; > + struct unknown_device unknown; > } dev; > char *id; > }; > @@ -239,6 +277,11 @@ struct domain { > > struct virt_device *dev_vcpu; > int dev_vcpu_ct; > + > + struct virt_device *dev_unknown; > + int dev_unknown_ct; > + > + struct others *others; > }; > > struct virt_device *virt_device_dup(struct virt_device *dev); > From jferlan at redhat.com Wed Nov 13 01:20:40 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 12 Nov 2013 20:20:40 -0500 Subject: [Libvirt-cim] [PATCH V2 02/48] Add others and unknown_device clean up In-Reply-To: <1382928377-16331-3-git-send-email-gesaint@linux.vnet.ibm.com> References: <1382928377-16331-1-git-send-email-gesaint@linux.vnet.ibm.com> <1382928377-16331-3-git-send-email-gesaint@linux.vnet.ibm.com> Message-ID: <5282D3E8.2090207@redhat.com> On 10/27/2013 10:45 PM, Xu Wang wrote: > Signed-off-by: Xu Wang > --- > libxkutil/device_parsing.c | 54 ++++++++++++++++++++++++++++++++++++++++++++ > 1 files changed, 54 insertions(+), 0 deletions(-) > If I apply the cimtest patches from Viktor, I get cimtest failures after applying this patch. That's about as far as I've had time to dig. It seems cimprovagt cores and for some reason the abrt isn't saving things... The failures are in: -------------------------------------------------------------------- RASDIndications - 02_guest_add_mod_rem_rasd_ind.py: FAIL ERROR - Exception details :(1, u'CIM_ERR_FAILED: Lost connection with cimprovagt "libvirt-cim".') ERROR - Exception: Unable to generate indication InvokeMethod(RemoveResourceSettings): CIM_ERR_FAILED: Lost connection with cimprovagt "libvirt-cim". -------------------------------------------------------------------- SettingsDefine - 01_forward.py: FAIL ERROR - 6 device insts != 7 RASD insts -------------------------------------------------------------------- SettingsDefine - 02_reverse.py: FAIL ERROR - u'KVM_ConsoleDisplayController' -------------------------------------------------------------------- VirtualSystemManagementService - 16_removeresource.py: FAIL ERROR - (1, u'CIM_ERR_FAILED: Lost connection with cimprovagt "libvirt-cim".') InvokeMethod(RemoveResourceSettings): CIM_ERR_FAILED: Lost connection with cimprovagt "libvirt-cim". -------------------------------------------------------------------- I'll investigate more in my morning.. FWIW: This is on a RHEL6.5 host that I'm borrowing... > diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c > index aecca4c..f1da880 100644 > --- a/libxkutil/device_parsing.c > +++ b/libxkutil/device_parsing.c > @@ -63,6 +63,49 @@ > /* Device parse function */ > typedef int (*dev_parse_func_t)(xmlNode *, struct virt_device **); > > +void cleanup_node_of_others(struct others *others) static void > +{ > + if (others == NULL) { > + return; > + } > + > + if (others->name) { > + free(others->name); > + } > + > + if (others->parent_name) { > + free(others->parent_name); > + } > + > + if (others->value) { > + free(others->value); > + } > + > + free(others); > +} > + > +void cleanup_others(struct others *others) static void > +{ > + struct others *head = others; > + > + if (others == NULL) > + return; Since we'd only enter the while loop if head (or others) were non NULL the above isn't really necessary > + > + while (head) { > + head = others->next; > + cleanup_node_of_others(others); > + others = head; > + } while (others) { next = others->next; cleanup_node_of_others(others); others = next; } Have to figure out why cimprovagt dies, I'm assuming it's in this logic somewhere... John > +} > + > +static void cleanup_unknown_device(struct unknown_device *dev) > +{ > + if (dev == NULL) > + return; > + > + cleanup_others(dev->others); > +} > + > static void cleanup_disk_device(struct disk_device *dev) > { > if (dev == NULL) > @@ -77,6 +120,7 @@ static void cleanup_disk_device(struct disk_device *dev) > free(dev->virtual_dev); > free(dev->bus_type); > free(dev->access_mode); > + cleanup_others(dev->others); > } > > static void cleanup_vsi_device(struct vsi_device *dev) > @@ -107,6 +151,7 @@ static void cleanup_net_device(struct net_device *dev) > free(dev->filter_ref); > free(dev->poolid); > cleanup_vsi_device(&dev->vsi); > + cleanup_others(dev->others); > } > > static void cleanup_emu_device(struct emu_device *dev) > @@ -115,6 +160,7 @@ static void cleanup_emu_device(struct emu_device *dev) > return; > > free(dev->path); > + cleanup_others(dev->others); > } > > static void cleanup_vnc_device(struct graphics_device *dev) > @@ -123,6 +169,7 @@ static void cleanup_vnc_device(struct graphics_device *dev) > free(dev->dev.vnc.host); > free(dev->dev.vnc.keymap); > free(dev->dev.vnc.passwd); > + cleanup_others(dev->dev.vnc.others); > } > > static void cleanup_sdl_device(struct graphics_device *dev) > @@ -130,6 +177,7 @@ static void cleanup_sdl_device(struct graphics_device *dev) > free(dev->dev.sdl.display); > free(dev->dev.sdl.xauth); > free(dev->dev.sdl.fullscreen); > + cleanup_others(dev->dev.sdl.others); > } > > static void cleanup_graphics_device(struct graphics_device *dev) > @@ -143,6 +191,7 @@ static void cleanup_graphics_device(struct graphics_device *dev) > cleanup_vnc_device(dev); > > free(dev->type); > + cleanup_others(dev->others); > } > > static void cleanup_path_device(struct path_device *dev) > @@ -228,6 +277,7 @@ static void cleanup_console_device(struct console_device *dev) > dev->source_type = 0; > free(dev->target_type); > memset(&dev->source_dev, 0, sizeof(dev->source_dev)); > + cleanup_others(dev->others); > }; > > static void console_device_dup(struct console_device *t, > @@ -286,6 +336,7 @@ static void cleanup_input_device(struct input_device *dev) > > free(dev->type); > free(dev->bus); > + cleanup_others(dev->others); > } > > void cleanup_virt_device(struct virt_device *dev) > @@ -305,6 +356,8 @@ void cleanup_virt_device(struct virt_device *dev) > cleanup_input_device(&dev->dev.input); > else if (dev->type == CIM_RES_TYPE_CONSOLE) > cleanup_console_device(&dev->dev.console); > + else if (dev->type == CIM_RES_TYPE_UNKNOWN) > + cleanup_unknown_device(&dev->dev.unknown); > > free(dev->id); > > @@ -1693,6 +1746,7 @@ void cleanup_dominfo(struct domain **dominfo) > cleanup_virt_devices(&dom->dev_graphics, dom->dev_graphics_ct); > cleanup_virt_devices(&dom->dev_input, dom->dev_input_ct); > cleanup_virt_devices(&dom->dev_console, dom->dev_console_ct); > + cleanup_virt_devices(&dom->dev_unknown, dom->dev_unknown_ct); > > free(dom); > > From gesaint at linux.vnet.ibm.com Wed Nov 13 03:27:36 2013 From: gesaint at linux.vnet.ibm.com (Xu Wang) Date: Wed, 13 Nov 2013 11:27:36 +0800 Subject: [Libvirt-cim] [PATCH V2 01/48] Add others member for saving unsupported tag and unknown device In-Reply-To: <5282CEDF.4090506@redhat.com> References: <1382928377-16331-1-git-send-email-gesaint@linux.vnet.ibm.com> <1382928377-16331-2-git-send-email-gesaint@linux.vnet.ibm.com> <5282CEDF.4090506@redhat.com> Message-ID: <5282F1A8.3@linux.vnet.ibm.com> ? 2013/11/13 8:59, John Ferlan ??: > On 10/27/2013 10:45 PM, Xu Wang wrote: >> Signed-off-by: Xu Wang >> --- >> libxkutil/device_parsing.h | 43 +++++++++++++++++++++++++++++++++++++++++++ >> 1 files changed, 43 insertions(+), 0 deletions(-) >> > > I am going to try to take these one at a time. Run them through cimtest > singly to see if I can flesh out errors. > > I didn't get very far - 2/48 causes some failures in cimtest > > ... > >> diff --git a/libxkutil/device_parsing.h b/libxkutil/device_parsing.h >> index 2803d6a..147551a 100644 >> --- a/libxkutil/device_parsing.h >> +++ b/libxkutil/device_parsing.h >> @@ -33,6 +33,33 @@ >> >> #include "../src/svpc_types.h" >> >> +/* The structure for saving unknown tag in the xml */ >> +enum others_type { >> + TYPE_PROP, >> + TYPE_NODE >> +}; >> + >> +struct others { > There's an extra space after the { which git am complains about > >> + /* To identify the different tags with same name */ >> + int id; >> + char *name; >> + int parent_id; >> + char *parent_name; >> + enum others_type type; >> + char *value; >> + struct others *next; >> + enum status { >> + ACTIVE, >> + INACTIVE >> + }; > Older compiler complains: > > ../libxkutil/device_parsing.h:54: warning: declaration does not declare > anything > > > New compiler where I don't ignore warnings stops: > > In file included from device_parsing.c:33:0: > device_parsing.h:54:10: error: declaration does not declare anything > [-Werror] > }; > ^ > cc1: all warnings being treated as errors > > I'll have to see more to determine whether I like this pointer to > pointer within structures... There's usually some assumption somewhere > which causes a failure... > > John Dear John, My idea is, using a data structure (a link list) to keep the elements left after all kinds of virtual devices fetched tags they needed from xml. And if I just mark a node of others with status instead of delete it, the whole structure of xml could be saved as well. The introduction of 'id' in 'others' structure is to identify several with the same name. The console device support new introduced and some tags in 'unknown_device' have to use 'id' to identify elements. It maybe a little coarse now so I need any suggestion from you to make it works better. If you have any better idea to solve the unsupported tags missing issue please share with me :-) Because so many patches about it merged and there will be more in the future. These 48 patches just the first part in my designing. After that I want to build the management about all data (even in 'others' member). And all of these updates are compatitable with older version libvirt-cim. So the upper layer need little change and keep the original feature but could make libvirt-cim becomes more powerful. XFAIL about hotplug disappeared after updated. About the warning you mentioned above I'll check and fix it in the new version. Thanks, Xu Wang >> +}; >> + >> +/* The structure for saving unknown device */ >> +struct unknown_device { >> + char *name; >> + struct others *others; >> +}; >> + >> struct vsi_device { >> char *vsi_type; >> char *manager_id; >> @@ -56,6 +83,7 @@ struct disk_device { >> char *bus_type; >> char *cache; >> char *access_mode; /* access modes for DISK_FS (filesystem) type */ >> + struct others *others; >> }; >> >> struct net_device { >> @@ -70,6 +98,7 @@ struct net_device { >> uint64_t reservation; >> uint64_t limit; >> struct vsi_device vsi; >> + struct others *others; >> }; >> >> struct mem_device { >> @@ -78,16 +107,19 @@ struct mem_device { >> enum { MEM_DUMP_CORE_NOT_SET, >> MEM_DUMP_CORE_ON, >> MEM_DUMP_CORE_OFF } dumpCore; >> + struct others *others; >> }; >> >> struct vcpu_device { >> uint64_t quantity; >> uint32_t weight; >> uint64_t limit; >> + struct others *others; >> }; >> >> struct emu_device { >> char *path; >> + struct others *others; >> }; >> >> struct vnc_device { >> @@ -95,12 +127,14 @@ struct vnc_device { >> char *host; >> char *keymap; >> char *passwd; >> + struct others *others; >> }; >> >> struct sdl_device { >> char *display; >> char *xauth; >> char *fullscreen; >> + struct others *others; >> }; >> >> struct graphics_device { >> @@ -109,6 +143,7 @@ struct graphics_device { >> struct vnc_device vnc; >> struct sdl_device sdl; >> } dev; >> + struct others *others; >> }; >> >> struct path_device { >> @@ -146,11 +181,13 @@ struct console_device { >> struct udp_device udp; >> } source_dev; >> char *target_type; >> + struct others *others; >> }; >> >> struct input_device { >> char *type; >> char *bus; >> + struct others *others; >> }; >> >> struct virt_device { >> @@ -164,6 +201,7 @@ struct virt_device { >> struct graphics_device graphics; >> struct console_device console; >> struct input_device input; >> + struct unknown_device unknown; >> } dev; >> char *id; >> }; >> @@ -239,6 +277,11 @@ struct domain { >> >> struct virt_device *dev_vcpu; >> int dev_vcpu_ct; >> + >> + struct virt_device *dev_unknown; >> + int dev_unknown_ct; >> + >> + struct others *others; >> }; >> >> struct virt_device *virt_device_dup(struct virt_device *dev); >> From mihajlov at linux.vnet.ibm.com Wed Nov 13 08:42:16 2013 From: mihajlov at linux.vnet.ibm.com (Viktor Mihajlovski) Date: Wed, 13 Nov 2013 09:42:16 +0100 Subject: [Libvirt-cim] [PATCH 0/2] cimtest support for consoles In-Reply-To: <5282C622.9060501@redhat.com> References: <1381489788-21481-1-git-send-email-mihajlov@linux.vnet.ibm.com> <5282C622.9060501@redhat.com> Message-ID: <52833B68.30608@linux.vnet.ibm.com> On 11/13/2013 01:21 AM, John Ferlan wrote: > On 10/11/2013 07:09 AM, Viktor Mihajlovski wrote: >> This series extends the KVM cimtests with the verification of >> the console instrumentation. >> If the tested libvirt-cim provider has the necessary release >> level, the test domains are instantiated with a console. >> The other patch deals with the fact that certain device types >> (like graphics) are supported on s390. >> >> I ran the tests on an x86 system as well seeing no regressions, >> but an independent test run would be welcome... >> >> Viktor Mihajlovski (2): >> cimtest: Adding support for ConsoleRASD >> cimtest: Fix s390 failures >> > > Not sure how to approach this series. On the one hand things look good; > however, when run with Xu Wang's 48 patch series it seems there's > assumptions in one set of patches or the other. > > In particular, there's numerous failures with the libvirt message "Only > the first console can be a serial port", which means for some reason > there's more than one console in the guest configured to use the serial > line. > > Since patch 1/2 deals with console's I have to assume that it's the > reason/cause for what I've seen. I haven't had the time to dig deep on > it though, but I guess I'd ask - is the code that seems to add a console > to a guest ever (or need to check) whether the guest already has one? > Would the 48 patch series somehow "assume" a need to put a console in an > others list. Just throwing random ideas out there. It's been a long day. > I'll have to check. It could be that with Xu Wang's patch we see tags be preserved which have been silently discarded before. Let's take one step at a time... -- Mit freundlichen Gr??en/Kind Regards Viktor Mihajlovski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martina K?deritz Gesch?ftsf?hrung: Dirk Wittkopp Sitz der Gesellschaft: B?blingen Registergericht: Amtsgericht Stuttgart, HRB 243294 From jferlan at redhat.com Wed Nov 13 15:56:26 2013 From: jferlan at redhat.com (John Ferlan) Date: Wed, 13 Nov 2013 10:56:26 -0500 Subject: [Libvirt-cim] [PATCH 1/2] cimtest: Adding support for ConsoleRASD In-Reply-To: <1381489788-21481-2-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1381489788-21481-1-git-send-email-mihajlov@linux.vnet.ibm.com> <1381489788-21481-2-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <5283A12A.8090105@redhat.com> On 10/11/2013 07:09 AM, Viktor Mihajlovski wrote: > The test domains are now defined with a ConsoleRASD if the > tested libvirt-cim does support those. A few testcases had > to be adjusted to properly handle the newly encountered instances > and associations. > > Signed-off-by: Viktor Mihajlovski > --- > .../cimtest/ElementSettingData/01_forward.py | 5 ++- > .../cimtest/HostSystem/02_hostsystem_to_rasd.py | 2 ++ > .../ResourceAllocationFromPool/01_forward.py | 3 ++ > .../ResourceAllocationFromPool/02_reverse.py | 3 ++ > .../libvirt-cim/cimtest/SystemDevice/01_forward.py | 5 +++ > suites/libvirt-cim/cimtest/VSSD/04_vssd_to_rasd.py | 11 ++++-- > .../02_reverse.py | 9 +++-- > suites/libvirt-cim/lib/XenKvmLib/devices.py | 6 ++++ > suites/libvirt-cim/lib/XenKvmLib/rasd.py | 21 ++++++++++- > suites/libvirt-cim/lib/XenKvmLib/vsms.py | 35 ++++++++++++++++++ > suites/libvirt-cim/lib/XenKvmLib/vxml.py | 38 ++++++++++++++------ > 11 files changed, 122 insertions(+), 16 deletions(-) > > diff --git a/suites/libvirt-cim/cimtest/ElementSettingData/01_forward.py b/suites/libvirt-cim/cimtest/ElementSettingData/01_forward.py > index 8c2a556..8b57b8f 100755 > --- a/suites/libvirt-cim/cimtest/ElementSettingData/01_forward.py > +++ b/suites/libvirt-cim/cimtest/ElementSettingData/01_forward.py > @@ -59,6 +59,7 @@ from XenKvmLib.const import do_main, get_provider_version > > sup_types = ['Xen', 'XenFV', 'KVM', 'LXC'] > input_graphics_pool_rev = 757 > +console_rev = 1276 > > test_dom = "esd_dom" > vmac = "00:11:22:33:44:aa" > @@ -148,7 +149,9 @@ def main(): > if curr_cim_rev >= input_graphics_pool_rev: > keys['InputResourceAllocationSettingData'] = input_device > keys['GraphicsResourceAllocationSettingData'] = "%s/vnc" % test_dom > - > + if curr_cim_rev >= console_rev: > + keys['ConsoleResourceAllocationSettingData'] = "%s/charconsole:0" % test_dom > + > ret = cxml.cim_define(options.ip) > if not ret: > logger.error("Failed to define the dom: %s", test_dom) > diff --git a/suites/libvirt-cim/cimtest/HostSystem/02_hostsystem_to_rasd.py b/suites/libvirt-cim/cimtest/HostSystem/02_hostsystem_to_rasd.py > index fe0f2cc..78433e7 100644 > --- a/suites/libvirt-cim/cimtest/HostSystem/02_hostsystem_to_rasd.py > +++ b/suites/libvirt-cim/cimtest/HostSystem/02_hostsystem_to_rasd.py > @@ -182,6 +182,8 @@ def verify_RASD_values(server, sd_assoc_info, vsxml, virt="Xen"): > status = verify_memrasd_values(assoc_info[index], rasd) > elif 'GraphicsResourceAllocationSettingData' in CCName: > status = verify_displayrasd_values(assoc_info[index], rasd) > + elif 'ConsoleResourceAllocationSettingData' in CCName: > + status = verify_displayrasd_values(assoc_info[index], rasd) > elif 'InputResourceAllocationSettingData' in CCName: > status = verify_inputrasd_values(assoc_info[index], rasd) > if status != PASS and virt == 'LXC': > diff --git a/suites/libvirt-cim/cimtest/ResourceAllocationFromPool/01_forward.py b/suites/libvirt-cim/cimtest/ResourceAllocationFromPool/01_forward.py > index a7d9c35..ec47c44 100644 > --- a/suites/libvirt-cim/cimtest/ResourceAllocationFromPool/01_forward.py > +++ b/suites/libvirt-cim/cimtest/ResourceAllocationFromPool/01_forward.py > @@ -75,6 +75,9 @@ def init_rasd_list(virt, ip, guest_name): > logger.error("Enum RASDs failed") > return rasd_insts, status > > + if rasds.has_key(get_typed_class(virt, 'ConsoleResourceAllocationSettingData')): > + del rasds[get_typed_class(virt, 'ConsoleResourceAllocationSettingData')] > + > for rasd_cn, rasd_list in rasds.iteritems(): > if virt == "LXC" and rasd_cn == disk_rasd_cn: > continue > diff --git a/suites/libvirt-cim/cimtest/ResourceAllocationFromPool/02_reverse.py b/suites/libvirt-cim/cimtest/ResourceAllocationFromPool/02_reverse.py > index bef114d..430d33f 100644 > --- a/suites/libvirt-cim/cimtest/ResourceAllocationFromPool/02_reverse.py > +++ b/suites/libvirt-cim/cimtest/ResourceAllocationFromPool/02_reverse.py > @@ -171,6 +171,9 @@ def main(): > if status != PASS: > raise Exception("Unable to build pool instance list") > > + if rasds.has_key(get_typed_class(options.virt, 'ConsoleResourceAllocationSettingData')): > + del rasds[get_typed_class(options.virt, 'ConsoleResourceAllocationSettingData')] > + > if len(rasds) != len(pools): > raise Exception("%d RASD insts != %d pool insts" % (len(rasds), > len(pools))) > diff --git a/suites/libvirt-cim/cimtest/SystemDevice/01_forward.py b/suites/libvirt-cim/cimtest/SystemDevice/01_forward.py > index 15a4ff5..f956621 100644 > --- a/suites/libvirt-cim/cimtest/SystemDevice/01_forward.py > +++ b/suites/libvirt-cim/cimtest/SystemDevice/01_forward.py > @@ -36,6 +36,7 @@ from CimTest.ReturnCodes import PASS, FAIL > > sup_types = ['Xen', 'KVM', 'XenFV', 'LXC'] > input_graphics_pool_rev = 757 > +console_dev_rev = 1276 > > test_dom = "test_domain" > test_mac = "00:11:22:33:44:55" > @@ -120,6 +121,10 @@ def main(): > graphics_cn = get_typed_class(virt, "DisplayController") > exp_pllist[graphics_cn] = ['%s/vnc' % test_dom] > > + if curr_cim_rev >= console_dev_rev: > + console_cn = get_typed_class(virt, "ConsoleDisplayController") > + exp_pllist[console_cn] = ['%s/charconsole:0' % test_dom] > + > > try: > res_pllist = {} > diff --git a/suites/libvirt-cim/cimtest/VSSD/04_vssd_to_rasd.py b/suites/libvirt-cim/cimtest/VSSD/04_vssd_to_rasd.py > index 04b0b6e..bdd91a8 100755 > --- a/suites/libvirt-cim/cimtest/VSSD/04_vssd_to_rasd.py > +++ b/suites/libvirt-cim/cimtest/VSSD/04_vssd_to_rasd.py > @@ -54,7 +54,7 @@ from XenKvmLib.classes import get_typed_class > from XenKvmLib import rasd > from XenKvmLib.rasd import verify_procrasd_values, verify_netrasd_values, \ > verify_diskrasd_values, verify_memrasd_values, verify_displayrasd_values, \ > -rasd_init_list, verify_inputrasd_values > +rasd_init_list, verify_inputrasd_values, verify_consolerasd_values > from XenKvmLib.const import default_network_name > > libvirt_bug = "00009" > @@ -173,6 +173,10 @@ def verify_rasd_values(rasd_values_info, server): > memrasd = rasd_values_list['%s' %in_list['mem']] > displayrasd = rasd_values_list['%s' %in_list['display']] > inputrasd = rasd_values_list['%s' %in_list['point']] > + if 'console' in in_list: > + consolerasd = rasd_values_list['%s' %in_list['console']] > + else: > + consolerasd = [] > > try: > for rasd_instance in rasd_values_info: > @@ -186,7 +190,10 @@ def verify_rasd_values(rasd_values_info, server): > elif 'MemResourceAllocationSettingData' in CCName : > status = verify_memrasd_values(rasd_instance, memrasd) > elif 'GraphicsResourceAllocationSettingData' in CCName : > - status = verify_displayrasd_values(rasd_instance, displayrasd) > + if not 'console' in rasd_instance['InstanceID']: > + status = verify_displayrasd_values(rasd_instance, displayrasd) > + elif 'ConsoleResourceAllocationSettingData' in CCName : > + status = verify_consolerasd_values(rasd_instance, consolerasd) Hmm... 'console', 'cons_rasd', 'charconsole:0' - it's all so confusing at least to me! Perhaps this is where the two consoles using serial is coming from. > elif 'InputResourceAllocationSettingData' in CCName: > status = verify_inputrasd_values(rasd_instance, inputrasd) > if status != PASS and virt== 'LXC': > diff --git a/suites/libvirt-cim/cimtest/VirtualSystemSettingDataComponent/02_reverse.py b/suites/libvirt-cim/cimtest/VirtualSystemSettingDataComponent/02_reverse.py > index 3a13b2a..24e8440 100644 > --- a/suites/libvirt-cim/cimtest/VirtualSystemSettingDataComponent/02_reverse.py > +++ b/suites/libvirt-cim/cimtest/VirtualSystemSettingDataComponent/02_reverse.py > @@ -95,7 +95,8 @@ def assoc_values(ip, assoc_info, virt="Xen"): > "disk_rasd" : '%s/%s' %(test_dom, test_disk), > "mem_rasd" : '%s/%s' %(test_dom, "mem"), > "input_rasd": '%s/%s' %(test_dom, input_device), > - "grap_rasd" : '%s/%s' %(test_dom, "vnc") > + "grap_rasd" : '%s/%s' %(test_dom, "vnc"), > + "cons_rasd" : '%s/%s' %(test_dom, "charconsole:0") > > } > > @@ -110,6 +111,7 @@ def assoc_values(ip, assoc_info, virt="Xen"): > mem_cn = get_typed_class(virt, 'MemResourceAllocationSettingData') > input_cn = get_typed_class(virt, 'InputResourceAllocationSettingData') > grap_cn = get_typed_class(virt, 'GraphicsResourceAllocationSettingData') > + cons_cn = get_typed_class(virt, 'ConsoleResourceAllocationSettingData') > > for inst in assoc_info: > if inst.classname == proc_cn: > @@ -132,8 +134,11 @@ def assoc_values(ip, assoc_info, virt="Xen"): > elif inst.classname == grap_cn: > status = check_rasd_values(inst['InstanceID'], > rasd_list['grap_rasd']) > + elif inst.classname == cons_cn: > + status = check_rasd_values(inst['InstanceID'], > + rasd_list['cons_rasd']) > else: > - logger.error("Unexpected RASD instance type" ) > + logger.error("Unexpected RASD instance type %s", inst.classname ) > status = FAIL > > if status == FAIL: > diff --git a/suites/libvirt-cim/lib/XenKvmLib/devices.py b/suites/libvirt-cim/lib/XenKvmLib/devices.py > index 0ddccbb..275acad 100755 > --- a/suites/libvirt-cim/lib/XenKvmLib/devices.py > +++ b/suites/libvirt-cim/lib/XenKvmLib/devices.py > @@ -33,6 +33,7 @@ from XenKvmLib.enumclass import EnumInstances > > graphics_dev_rev = 725 > input_dev_rev = 745 > +console_dev_rev = 1276 > > def get_class(classname): > return eval(classname) > @@ -95,6 +96,8 @@ def dev_cn_to_rasd_cn(dev_cn, virt): > return get_typed_class(virt, "DiskResourceAllocationSettingData") > elif dev_cn.find('Memory') >= 0: > return get_typed_class(virt, "MemResourceAllocationSettingData") > + elif dev_cn.find('ConsoleDisplayController') >= 0: > + return get_typed_class(virt, "ConsoleResourceAllocationSettingData") > elif dev_cn.find('DisplayController') >= 0: > return get_typed_class(virt, "GraphicsResourceAllocationSettingData") > elif dev_cn.find('PointingDevice') >= 0: > @@ -112,6 +115,9 @@ def enum_dev(virt, ip): > if curr_cim_rev >= input_dev_rev: > dev_list.append('PointingDevice') > > + if curr_cim_rev >= console_dev_rev: > + dev_list.append('ConsoleDisplayController') > + So this will I think upset some test which says it expects 6 device but got 7 (I forget which one). > dev_insts = {} > > try: > diff --git a/suites/libvirt-cim/lib/XenKvmLib/rasd.py b/suites/libvirt-cim/lib/XenKvmLib/rasd.py > index 4d4240a..3675015 100644 > --- a/suites/libvirt-cim/lib/XenKvmLib/rasd.py > +++ b/suites/libvirt-cim/lib/XenKvmLib/rasd.py > @@ -50,9 +50,11 @@ memcn = 'Memory' > netcn = 'NetworkPort' > diskcn = 'LogicalDisk' > dccn = 'DisplayController' > +cccn = 'ConsoleDisplayController' > pdcn = 'PointingDevice' > > libvirt_rasd_storagepool_changes = 934 > +vsms_console_sup = 1276 > > def rasd_init_list(vsxml, virt, t_disk, t_dom, t_mac, t_mem, server): > """ > @@ -65,6 +67,7 @@ def rasd_init_list(vsxml, virt, t_disk, t_dom, t_mac, t_mem, server): > disk_cn = get_typed_class(virt, diskcn) > dc_cn = get_typed_class(virt, dccn) > pd_cn = get_typed_class(virt, pdcn) > + cc_cn = None > > in_list = { 'proc' : proc_cn, > 'mem' : mem_cn, > @@ -75,6 +78,11 @@ def rasd_init_list(vsxml, virt, t_disk, t_dom, t_mac, t_mem, server): > } > try: > > + curr_cim_rev, changeset = get_provider_version(virt, server) > + if curr_cim_rev >= vsms_console_sup: > + cc_cn = get_typed_class(virt, cccn) > + in_list['console'] = cc_cn > + > disk_path = vsxml.xml_get_disk_source() > if virt == 'LXC': > disk_path = '/var/lib/libvirt/images/lxc_files' > @@ -116,7 +124,11 @@ def rasd_init_list(vsxml, virt, t_disk, t_dom, t_mac, t_mem, server): > pd_cn : { > "InstanceID" : point_device > } > - } > + } > + if cc_cn != None: > + rasd_values[cc_cn] = { > + "InstanceID" : "%s/%s" %(t_dom, "charconsole:0") > + } Hmm.. no setting of the ResourceType value (nor anything else). Furthermore it seems that rather than hardcoded numbers, then RASD_TYPE_* constants should be used from XenKvmLib.vsms, right? That includes the settings in the fields above. It certainly would make things more clear... > except Exception, details: > logger.error("Exception: In fn rasd_init_list %s", details) > return FAIL, rasd_values, in_list > @@ -150,6 +162,13 @@ def verify_displayrasd_values(assoc_info, displayrasd_list): > status = FAIL > return status > > +def verify_consolerasd_values(assoc_info, consolerasd_list): > + status = PASS > + if assoc_info['InstanceID'] != consolerasd_list['InstanceID']: > + InstId_err(assoc_info, consolerasd_list) > + status = FAIL > + return status > + > def verify_inputrasd_values(assoc_info, inputrasd_list): > status = PASS > if assoc_info['InstanceID'] != inputrasd_list['InstanceID']: > diff --git a/suites/libvirt-cim/lib/XenKvmLib/vsms.py b/suites/libvirt-cim/lib/XenKvmLib/vsms.py > index d7f33f9..955026f 100755 > --- a/suites/libvirt-cim/lib/XenKvmLib/vsms.py > +++ b/suites/libvirt-cim/lib/XenKvmLib/vsms.py > @@ -28,6 +28,7 @@ from CimTest import Globals > from XenKvmLib import const > from XenKvmLib.classes import get_typed_class, get_class_type, virt_types > > +RASD_TYPE_OTHER = 1 Should this be 32769? I'm assuming they roughly follow CIM_RES_TYPE* values in libvirt-cim > RASD_TYPE_PROC = 3 > RASD_TYPE_MEM = 4 > RASD_TYPE_NET_ETHER = 10 > @@ -259,6 +260,40 @@ class LXC_MemResourceAllocationSettingData(CIM_MemResourceAllocationSettingData) > def get_masd_class(virt): > pass > > +class CIM_ConsoleResourceAllocationSettingData(CIMClassMOF): > + def __init__(self, name, source_type=2, target_type="virtio", > + source_path=None, connect_URL=None, bind_URL=None): > + self.InstanceID = '%s/charconsole:0' %name > + self.ResourceType = RASD_TYPE_OTHER > + self.OtherResourceType = 'console' > + > + if source_type != None: > + self.SourceType = source_type source_type defaults to 2 or passed generally as 0, so None probably doesn't work here, right? > + > + if target_type != None: > + self.TargetType = target_type target_type defaults to "virtio", so is None ever possible? > + > + if source_path != None: > + self.SourcePath = source_path > + > + if bind_URL != None: > + self.BindURL = bind_URL > + > + if connect_URL != None: > + self.ConnectURL = connect_URL > + > +class Xen_ConsoleResourceAllocationSettingData(CIM_ConsoleResourceAllocationSettingData): > + pass > + > +class KVM_ConsoleResourceAllocationSettingData(CIM_ConsoleResourceAllocationSettingData): > + pass > + > +class LXC_ConsoleResourceAllocationSettingData(CIM_ConsoleResourceAllocationSettingData): > + pass > + > + at eval_cls('ConsoleResourceAllocationSettingData') > +def get_casd_class(virt): > + pass > > class CIM_GraphicsResourceAllocationSettingData(CIMClassMOF): > def __init__(self, name, res_sub_type="vnc", ip=None, ipv6_flag=None, > diff --git a/suites/libvirt-cim/lib/XenKvmLib/vxml.py b/suites/libvirt-cim/lib/XenKvmLib/vxml.py > index 74c4f23..9489a5c 100644 > --- a/suites/libvirt-cim/lib/XenKvmLib/vxml.py > +++ b/suites/libvirt-cim/lib/XenKvmLib/vxml.py > @@ -50,9 +50,11 @@ from CimTest.ReturnCodes import SKIP, PASS, FAIL > from XenKvmLib.classes import virt_types, get_typed_class > from XenKvmLib.enumclass import GetInstance > from XenKvmLib.const import get_provider_version > +from XenKvmLib.xm_virt_util import host_cpu_model > > vsms_graphics_sup = 763 > vsms_inputdev_sup = 771 > +vsms_console_sup = 1276 > > class XMLClass: > xml_string = "" > @@ -598,7 +600,8 @@ class VirtCIM: > def __init__(self, virt, dom_name, uuid, pae, acpi, apic, disk_dev, > disk_source, net_type, net_name, net_mac, vcpus, mem, > mem_allocunits, emu_type, grstype, ip, > - is_ipv6_only, port_num, kmap, irstype, btype, vnc_passwd): > + is_ipv6_only, port_num, kmap, irstype, btype, vnc_passwd, > + con_source_type, con_target_type): > self.virt = virt > self.domain_name = dom_name > self.err_rc = None > @@ -623,6 +626,9 @@ class VirtCIM: > ipv6_flag=is_ipv6_only, > lport=port_num, keymap=kmap, > vnc_passwd=vnc_passwd) > + self.casd = vsms.get_casd_class(virt)(name=dom_name, > + source_type=con_source_type, > + target_type=con_target_type) > self.masd = vsms.get_masd_class(virt)(megabytes=mem, > mallocunits=mem_allocunits, > name=dom_name) > @@ -655,9 +661,14 @@ class VirtCIM: > res_settings.append(str(self.gasd)) > > if curr_cim_rev >= vsms_inputdev_sup: > - if self.iasd is not None: > + if self.iasd is not None and \ > + not str(host_cpu_model(ip, self.virt)).startswith('s390'): This probably should be in patch 2/2... > res_settings.append(str(self.iasd)) > > + if curr_cim_rev >= vsms_console_sup: > + if ref_conf is None and self.casd is not None: > + res_settings.append(str(self.casd)) > + > if ref_conf is None: > ref_conf = ' ' > > @@ -808,6 +819,8 @@ class VirtCIM: > self.dasd = rasd > elif cn.find("NetResourceAllocationSettingData") >= 0: > self.nasd = rasd > + elif cn.find("ConsoleResourceAllocationSettingData") >=0: > + self.casd = rasd > elif cn.find("GraphicsResourceAllocationSettingData") >= 0: > self.gasd = rasd > > @@ -849,7 +862,8 @@ class XenXML(VirtXML, VirtCIM): > emu_type=None, grstype="vnc", address=None, > is_ipv6_only=None, > port_num='-1', keymap="en-us", irstype="mouse", > - btype="xen", vnc_passwd=None): > + btype="xen", vnc_passwd=None, > + con_source_type=0, con_target_type="virtio"): > if not (os.path.exists(const.Xen_kernel_path) \ > and os.path.exists(const.Xen_init_path)): > logger.error('ERROR: Either the kernel image ' > @@ -863,7 +877,7 @@ class XenXML(VirtXML, VirtCIM): > disk_file_path, ntype, net_name, mac, vcpus, mem, > mem_allocunits, emu_type, grstype, address, > is_ipv6_only, port_num, keymap, irstype, btype, > - vnc_passwd) > + vnc_passwd, con_source_type, con_target_type) > > def _os(self, os_kernel, os_initrd): > os = self.get_node('/domain/os') > @@ -920,7 +934,8 @@ class KVMXML(VirtXML, VirtCIM): > emu_type=None, grstype="vnc", address=None, > is_ipv6_only=None, > port_num='-1', keymap="en-us", irstype="mouse", > - btype="ps2", vnc_passwd=None): > + btype="ps2", vnc_passwd=None, > + con_source_type=0, con_target_type="virtio"): > if not os.path.exists(disk_file_path): > logger.error('Error: Disk image %s does not exist', disk_file_path) > sys.exit(1) > @@ -929,7 +944,7 @@ class KVMXML(VirtXML, VirtCIM): > disk_file_path, ntype, net_name, mac, vcpus, mem, > mem_allocunits, emu_type, grstype, address, > is_ipv6_only, port_num, keymap, irstype, btype, > - vnc_passwd) > + vnc_passwd, con_source_type, con_target_type) > self._os() > self._devices(const.KVM_default_emulator, ntype, > disk_file_path, disk, mac, net_name) > @@ -983,7 +998,8 @@ class XenFVXML(VirtXML, VirtCIM): > emu_type=None, grstype="vnc", > address=None, is_ipv6_only=None, port_num='-1', > keymap="en-us", > - irstype="mouse", btype="ps2", vnc_passwd=None): > + irstype="mouse", btype="ps2", vnc_passwd=None, > + con_source_type=0, con_target_type="virtio"): > if not os.path.exists(disk_file_path): > logger.error('Error: Disk image %s does not exist', disk_file_path) > sys.exit(1) > @@ -992,7 +1008,8 @@ class XenFVXML(VirtXML, VirtCIM): > disk_file_path, ntype, net_name, mac, vcpus, mem, > mem_allocunits, emu_type, grstype, address, > is_ipv6_only, port_num, > - keymap, irstype, btype, vnc_passwd) > + keymap, irstype, btype, vnc_passwd, > + con_source_type, con_target_type) > self._os(const.XenFV_default_loader) > self._devices(const.XenFV_default_emulator, > ntype, mac, net_name, disk_file_path, disk) > @@ -1036,7 +1053,8 @@ class LXCXML(VirtXML, VirtCIM): > tty=const.LXC_default_tty, grstype="vnc", > address=None, is_ipv6_only=None, port_num='-1', > keymap="en-us", > - irstype="mouse", btype="usb", vnc_passwd=None): > + irstype="mouse", btype="usb", vnc_passwd=None, > + con_source_type=0, con_target_type="virtio"): > VirtXML.__init__(self, 'lxc', test_dom, set_uuid(), mem, vcpus) > # pae, acpi and apic parameters doesn't make sense here, so we > # statically set them to False (a.k.a. ignore them) > @@ -1045,7 +1063,7 @@ class LXCXML(VirtXML, VirtCIM): > ntype, net_name, mac, vcpus, mem, > const.default_mallocunits, None, grstype, > address, is_ipv6_only, port_num, keymap, irstype, > - btype, vnc_passwd) > + btype, vnc_passwd, con_source_type, con_target_type) > self._os(const.LXC_init_path) > self._devices(const.LXC_default_emulator, mac, ntype, net_name, const.LXC_default_tty) > self.create_lxc_file(CIM_IP, const.LXC_init_path) > From mihajlov at linux.vnet.ibm.com Wed Nov 13 16:24:30 2013 From: mihajlov at linux.vnet.ibm.com (Viktor Mihajlovski) Date: Wed, 13 Nov 2013 17:24:30 +0100 Subject: [Libvirt-cim] [PATCH 2/5] libxkutil: Support for device addresses In-Reply-To: <5282ACE1.1080607@redhat.com> References: <1381764584-18025-1-git-send-email-mihajlov@linux.vnet.ibm.com> <1381764584-18025-3-git-send-email-mihajlov@linux.vnet.ibm.com> <5282ACE1.1080607@redhat.com> Message-ID: <5283A7BE.7010308@linux.vnet.ibm.com> On 11/12/2013 11:34 PM, John Ferlan wrote: [...] >> +int add_device_address_property(struct device_address *devaddr, >> + const char *key, >> + const char *value) >> +{ >> + char *k = NULL; >> + char *v = NULL; >> + char **list = NULL; >> + >> + if (key != NULL && value != NULL) { >> + k = strdup(key); >> + v = strdup(value); >> + if (k == NULL || v == NULL) >> + goto err; >> + >> + list = realloc(devaddr->key, sizeof(char*) * (devaddr->ct+1)); >> + if (list == NULL) >> + goto err; >> + devaddr->key = list; >> + >> + list = realloc(devaddr->value, sizeof(char*) * (devaddr->ct+1)); >> + if (list == NULL) > > There's a leak here since 'ct' isn't incremented until later, > devaddr->key will have the 'list' from above and that'll either be lost > or overwritten. I have to admit that you're not the first reviewer I have confused with this approach ... so: If realloc of the 'value' list fails, the devaddr content remains valid and unchanged, exactly because we don't increment the counter. And the intent is to preserve the passed-in devaddr even if adding new key-value pairs fails. It's true that the 'key' list will have an excess element, which doesn't hurt as it doesn't need to be freed (it has not been set). Frankly, I don't have a better idea, unless not to use realloc but do malloc's followed by memcpy's instead. > > I see two ways out... > free(devaddr->key); > devaddr->key = NULL; > > or let err: handle the free(list) below... > > list = devaddr->key; > devaddr->key = NULL; > > Of course there's also list1 and list2... > > Let me know which you prefer, I can make the change before pushing... > > > John > >> + goto err; >> + devaddr->value = list; >> + >> + devaddr->key[devaddr->ct] = k; >> + devaddr->value[devaddr->ct] = v; >> + devaddr->ct += 1; >> + return 1; >> + } >> + >> + err: >> + free(k); >> + free(v); >> + free(list); >> + return 0; >> +} >> + >> + >> +static int parse_device_address(xmlNode *anode, struct device_address *devaddr) >> +{ >> + xmlAttr *attr = NULL; >> + char *name = NULL; >> + char *value = NULL; >> + >> + for (attr = anode->properties; attr != NULL; attr = attr->next) { >> + name = (char*) attr->name; >> + value = get_attr_value(anode, name); >> + if (!add_device_address_property(devaddr, name, value)) >> + goto err; >> + free(value); >> + } >> + >> + return 1; >> + >> + err: >> + cleanup_device_address(devaddr); >> + free(value); >> + >> + return 0; >> +} >> + >> static int parse_fs_device(xmlNode *dnode, struct virt_device **vdevs) >> { >> struct virt_device *vdev = NULL; >> @@ -386,6 +465,8 @@ static int parse_fs_device(xmlNode *dnode, struct virt_device **vdevs) >> } >> } else if (XSTREQ(child->name, "driver")) { >> ddev->driver_type = get_attr_value(child, "type"); >> + } else if (XSTREQ(child->name, "address")) { >> + parse_device_address(child, &ddev->address); >> } >> } >> >> @@ -459,6 +540,8 @@ static int parse_block_device(xmlNode *dnode, struct virt_device **vdevs) >> ddev->readonly = true; >> } else if (XSTREQ(child->name, "shareable")) { >> ddev->shareable = true; >> + } else if (XSTREQ(child->name, "address")) { >> + parse_device_address(child, &ddev->address); >> } >> } >> >> @@ -598,6 +681,8 @@ static int parse_net_device(xmlNode *inode, struct virt_device **vdevs) >> ndev->filter_ref = get_attr_value(child, "filter"); >> } else if (XSTREQ(child->name, "virtualport")) { >> parse_vsi_device(child, ndev); >> + } else if (XSTREQ(child->name, "address")) { >> + parse_device_address(child, &ndev->address); >> #if LIBVIR_VERSION_NUMBER >= 9000 >> } else if (XSTREQ(child->name, "bandwidth")) { >> /* Network QoS bandwidth support */ >> @@ -1167,6 +1252,32 @@ static int parse_devices(const char *xml, struct virt_device **_list, int type) >> return count; >> } >> >> +static void duplicate_device_address(struct device_address *to, const struct device_address *from) >> +{ >> + int i; >> + >> + if (from == NULL || to == NULL || from->ct == 0) >> + return; >> + >> + to->ct = from->ct; >> + to->key = calloc(from->ct, sizeof(char*)); >> + to->value = calloc(from->ct, sizeof(char*)); >> + if (to->key == NULL || to->value == NULL) >> + goto err; >> + >> + for (i = 0; i < from->ct; i++) { >> + to->key[i] = strdup(from->key[i]); >> + to->value[i] = strdup(from->value[i]); >> + if (to->key[i] == NULL || to->value[i] == NULL) >> + goto err; >> + } >> + >> + return; >> + >> + err: >> + cleanup_device_address(to); >> +} >> + >> struct virt_device *virt_device_dup(struct virt_device *_dev) >> { >> struct virt_device *dev; >> @@ -1196,6 +1307,7 @@ struct virt_device *virt_device_dup(struct virt_device *_dev) >> DUP_FIELD(dev, _dev, dev.net.vsi.profile_id); >> dev->dev.net.reservation = _dev->dev.net.reservation; >> dev->dev.net.limit = _dev->dev.net.limit; >> + duplicate_device_address(&dev->dev.net.address, &_dev->dev.net.address); >> } else if (dev->type == CIM_RES_TYPE_DISK) { >> DUP_FIELD(dev, _dev, dev.disk.type); >> DUP_FIELD(dev, _dev, dev.disk.device); >> @@ -1209,6 +1321,7 @@ struct virt_device *virt_device_dup(struct virt_device *_dev) >> dev->dev.disk.disk_type = _dev->dev.disk.disk_type; >> dev->dev.disk.readonly = _dev->dev.disk.readonly; >> dev->dev.disk.shareable = _dev->dev.disk.shareable; >> + duplicate_device_address(&dev->dev.disk.address, &_dev->dev.disk.address); >> } else if (dev->type == CIM_RES_TYPE_MEM) { >> dev->dev.mem.size = _dev->dev.mem.size; >> dev->dev.mem.maxsize = _dev->dev.mem.maxsize; >> diff --git a/libxkutil/device_parsing.h b/libxkutil/device_parsing.h >> index 4beac5c..92427c1 100644 >> --- a/libxkutil/device_parsing.h >> +++ b/libxkutil/device_parsing.h >> @@ -33,6 +33,12 @@ >> >> #include "../src/svpc_types.h" >> >> +struct device_address { >> + uint32_t ct; >> + char **key; >> + char **value; >> +}; >> + >> struct vsi_device { >> char *vsi_type; >> char *manager_id; >> @@ -56,6 +62,7 @@ struct disk_device { >> char *bus_type; >> char *cache; >> char *access_mode; /* access modes for DISK_FS (filesystem) type */ >> + struct device_address address; >> }; >> >> struct net_device { >> @@ -70,6 +77,7 @@ struct net_device { >> uint64_t reservation; >> uint64_t limit; >> struct vsi_device vsi; >> + struct device_address address; >> }; >> >> struct mem_device { >> @@ -257,6 +265,9 @@ int get_devices(virDomainPtr dom, struct virt_device **list, int type, >> void cleanup_virt_device(struct virt_device *dev); >> void cleanup_virt_devices(struct virt_device **devs, int count); >> >> +int add_device_address_property(struct device_address *devaddr, >> + const char *key, const char *value); >> + >> char *get_node_content(xmlNode *node); >> char *get_attr_value(xmlNode *node, char *attrname); >> >> diff --git a/libxkutil/xmlgen.c b/libxkutil/xmlgen.c >> index 7e8801d..40e2905 100644 >> --- a/libxkutil/xmlgen.c >> +++ b/libxkutil/xmlgen.c >> @@ -178,6 +178,26 @@ static const char *console_xml(xmlNodePtr root, struct domain *dominfo) >> BAD_CAST cdev->target_type); >> } >> } >> + >> + return NULL; >> +} >> + >> +static char *device_address_xml(xmlNodePtr root, struct device_address *addr) >> +{ >> + int i; >> + xmlNodePtr address; >> + >> + if (addr == NULL || addr->ct == 0) >> + return NULL; >> + >> + address = xmlNewChild(root, NULL, BAD_CAST "address", NULL); >> + if (address == NULL) >> + return XML_ERROR; >> + >> + for (i = 0; i < addr->ct; i++) { >> + xmlNewProp(address, BAD_CAST addr->key[i] , BAD_CAST addr->value[i]); >> + } >> + >> return NULL; >> } >> >> @@ -225,6 +245,9 @@ static char *disk_block_xml(xmlNodePtr root, struct disk_device *dev) >> if (dev->shareable) >> xmlNewChild(disk, NULL, BAD_CAST "shareable", NULL); >> >> + if (dev->address.ct > 0) >> + return device_address_xml(disk, &dev->address); >> + >> return NULL; >> } >> >> @@ -279,6 +302,8 @@ static const char *disk_file_xml(xmlNodePtr root, struct disk_device *dev) >> if (dev->shareable) >> xmlNewChild(disk, NULL, BAD_CAST "shareable", NULL); >> >> + if (dev->address.ct > 0) >> + return device_address_xml(disk, &dev->address); >> >> return NULL; >> } >> @@ -520,6 +545,9 @@ static const char *net_xml(xmlNodePtr root, struct domain *dominfo) >> } >> #endif >> >> + if (dev->dev.net.address.ct > 0) >> + msg = device_address_xml(nic, &dev->dev.net.address); >> + >> if (STREQ(dev->dev.net.type, "network")) { >> msg = set_net_source(nic, net, "network"); >> } else if (STREQ(dev->dev.net.type, "bridge")) { >> > > _______________________________________________ > Libvirt-cim mailing list > Libvirt-cim at redhat.com > https://www.redhat.com/mailman/listinfo/libvirt-cim > -- Mit freundlichen Gr??en/Kind Regards Viktor Mihajlovski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martina K?deritz Gesch?ftsf?hrung: Dirk Wittkopp Sitz der Gesellschaft: B?blingen Registergericht: Amtsgericht Stuttgart, HRB 243294 From jferlan at redhat.com Wed Nov 13 16:44:43 2013 From: jferlan at redhat.com (John Ferlan) Date: Wed, 13 Nov 2013 11:44:43 -0500 Subject: [Libvirt-cim] [PATCH 2/5] libxkutil: Support for device addresses In-Reply-To: <5283A7BE.7010308@linux.vnet.ibm.com> References: <1381764584-18025-1-git-send-email-mihajlov@linux.vnet.ibm.com> <1381764584-18025-3-git-send-email-mihajlov@linux.vnet.ibm.com> <5282ACE1.1080607@redhat.com> <5283A7BE.7010308@linux.vnet.ibm.com> Message-ID: <5283AC7B.6050708@redhat.com> On 11/13/2013 11:24 AM, Viktor Mihajlovski wrote: > On 11/12/2013 11:34 PM, John Ferlan wrote: > [...] >>> +int add_device_address_property(struct device_address *devaddr, >>> + const char *key, >>> + const char *value) >>> +{ >>> + char *k = NULL; >>> + char *v = NULL; >>> + char **list = NULL; >>> + >>> + if (key != NULL && value != NULL) { >>> + k = strdup(key); >>> + v = strdup(value); >>> + if (k == NULL || v == NULL) >>> + goto err; >>> + >>> + list = realloc(devaddr->key, sizeof(char*) * (devaddr->ct+1)); >>> + if (list == NULL) >>> + goto err; >>> + devaddr->key = list; >>> + >>> + list = realloc(devaddr->value, sizeof(char*) * (devaddr->ct+1)); >>> + if (list == NULL) >> >> There's a leak here since 'ct' isn't incremented until later, >> devaddr->key will have the 'list' from above and that'll either be lost >> or overwritten. > I have to admit that you're not the first reviewer I have > confused with this approach ... so: > If realloc of the 'value' list fails, the devaddr content > remains valid and unchanged, exactly because we don't increment > the counter. And the intent is to preserve the passed-in devaddr > even if adding new key-value pairs fails. > It's true that the 'key' list will have an excess element, which > doesn't hurt as it doesn't need to be freed (it has not been > set). > Frankly, I don't have a better idea, unless not to use > realloc but do malloc's followed by memcpy's instead. oh - right... The next time realloc comes along it could essentially realloc something of the same size which I supposed is a noop this is why I like the libvirt macros around memory (re)allocation and list manipulation. I never have to think about them... I'll push this series in a bit... Tks, John From jferlan at redhat.com Wed Nov 13 16:50:25 2013 From: jferlan at redhat.com (John Ferlan) Date: Wed, 13 Nov 2013 11:50:25 -0500 Subject: [Libvirt-cim] [RESEND PATCH] libvirt-cim: Changed resource type value EMU In-Reply-To: <1384273723-12730-1-git-send-email-daniel.hansel@linux.vnet.ibm.com> References: <1384273723-12730-1-git-send-email-daniel.hansel@linux.vnet.ibm.com> Message-ID: <5283ADD1.3060302@redhat.com> On 11/12/2013 11:28 AM, Daniel Hansel wrote: > To avoid conflicts with CIM resource type OTHER the value of the > resource type EMU is changed. > > Signed-off-by: Daniel Hansel > --- > src/svpc_types.h | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > This is now pushed Thanks, John From mihajlov at linux.vnet.ibm.com Wed Nov 13 16:51:20 2013 From: mihajlov at linux.vnet.ibm.com (Viktor Mihajlovski) Date: Wed, 13 Nov 2013 17:51:20 +0100 Subject: [Libvirt-cim] [PATCH 2/4] FilterEntry: Fix endianness issues In-Reply-To: <52829D8D.7040805@redhat.com> References: <1381492055-11502-1-git-send-email-mihajlov@linux.vnet.ibm.com> <1381492055-11502-3-git-send-email-mihajlov@linux.vnet.ibm.com> <52829D8D.7040805@redhat.com> Message-ID: <5283AE08.2030806@linux.vnet.ibm.com> On 11/12/2013 10:28 PM, John Ferlan wrote: > On 10/11/2013 07:47 AM, Viktor Mihajlovski wrote: >> From: Thilo Boehm >> >> A number of CIM properties was set in an endianness-unsafe manner >> leading to failures on big endian systems. >> >> Signed-off-by: Thilo Boehm >> Signed-off-by: Viktor Mihajlovski >> --- >> src/Virt_FilterEntry.c | 46 ++++++++++++++++++++++++---------------------- >> 1 file changed, 24 insertions(+), 22 deletions(-) >> >> diff --git a/src/Virt_FilterEntry.c b/src/Virt_FilterEntry.c >> index b7042da..e41b4b6 100644 >> --- a/src/Virt_FilterEntry.c >> +++ b/src/Virt_FilterEntry.c >> @@ -59,8 +59,8 @@ struct rule_data_t { >> const char *dstportend; >> }; >> >> -static int octets_from_mac(const char * s, unsigned int *buffer, >> - unsigned int size) >> +static int octets_from_mac(const char * s, uint8_t *buffer, >> + unsigned int size) >> { >> unsigned int _buffer[6]; >> unsigned int i, n = 0; >> @@ -86,8 +86,8 @@ static int octets_from_mac(const char * s, unsigned int *buffer, >> return n; >> } >> >> -static int octets_from_ip(const char * s, unsigned int *buffer, >> - unsigned int size) >> +static int octets_from_ip(const char * s, uint8_t *buffer, >> + unsigned int size) >> { >> struct in6_addr addr; >> unsigned int family = 0; >> @@ -116,7 +116,8 @@ static int octets_from_ip(const char * s, unsigned int *buffer, >> return n; >> } >> >> -static CMPIArray *octets_to_cmpi(const CMPIBroker *broker, unsigned int *bytes, int size) >> +static CMPIArray *octets_to_cmpi(const CMPIBroker *broker, uint8_t *bytes, >> + int size) >> { >> CMPIStatus s = {CMPI_RC_OK, NULL}; >> CMPIArray *array = NULL; >> @@ -216,7 +217,7 @@ static int convert_action(const char *s) >> return action; >> } >> >> -static unsigned long convert_protocol_id(const char *s) >> +static uint16_t convert_protocol_id(const char *s) >> { >> enum {NONE = 0, IPV4 = 2048, ARP = 2054, RARP = 32821, IPV6 = 34525} id = NONE; >> >> @@ -239,7 +240,7 @@ static void convert_mac_rule_to_instance( >> CMPIInstance *inst, >> const CMPIBroker *broker) >> { >> - unsigned int bytes[48]; >> + uint8_t bytes[48]; >> unsigned int size = 0; >> CMPIArray *array = NULL; >> >> @@ -280,7 +281,7 @@ static void convert_mac_rule_to_instance( >> (CMPIValue *)&array, CMPI_uint8A); >> >> if (rule->var.mac.protocol_id != NULL) { >> - unsigned long n = convert_protocol_id(rule->var.mac.protocol_id); >> + uint16_t n = convert_protocol_id(rule->var.mac.protocol_id); >> >> /* Unknown protocolid string. Try converting from hexadecimal value */ >> if (n == 0) >> @@ -366,18 +367,19 @@ static void convert_ip_rule_to_instance( >> CMPIInstance *inst, >> const CMPIBroker *broker) >> { >> - unsigned int bytes[48]; >> + uint8_t bytes[48]; >> unsigned int size = 0; >> - unsigned int n = 0; >> + uint8_t ipver_num = 0; >> + uint16_t port_num = 0; >> CMPIArray *array = NULL; >> struct rule_data_t rule_data; >> >> if (strstr(rule->protocol_id, "v6")) >> - n = 6; >> + ipver_num = 6; >> else >> - n = 4; >> + ipver_num = 4; >> >> - CMSetProperty(inst, "HdrIPVersion",(CMPIValue *)&n, CMPI_uint8); >> + CMSetProperty(inst, "HdrIPVersion",(CMPIValue *)&ipver_num, CMPI_uint8); >> >> fill_rule_data(rule, &rule_data); >> >> @@ -480,27 +482,27 @@ static void convert_ip_rule_to_instance( >> } >> >> if (rule_data.srcportstart) { >> - n = atoi(rule_data.srcportstart); >> + port_num = atoi(rule_data.srcportstart); > > Hmm.. technically atoi() returns an 'int'... In libvirt code this would > be a call to a function which does a strtoul() and makes sure there's no > overflow, etc. However, since this code is full of other situations > such as this, I'm "OK" with the way it's done... too true ... we have tried to restrict the patch to the big/little- endian fix and not to mix in cleanups. > > >> CMSetProperty(inst, "HdrSrcPortStart", >> - (CMPIValue *)&n, CMPI_uint16); >> + (CMPIValue *)&port_num, CMPI_uint16); >> } >> >> if (rule_data.srcportend) { >> - n = atoi(rule_data.srcportend); >> + port_num = atoi(rule_data.srcportend); >> CMSetProperty(inst, "HdrSrcPortEnd", >> - (CMPIValue *)&n, CMPI_uint16); >> + (CMPIValue *)&port_num, CMPI_uint16); >> } >> >> if (rule_data.dstportstart) { >> - n = atoi(rule_data.dstportstart); >> + port_num = atoi(rule_data.dstportstart); >> CMSetProperty(inst, "HdrDestPortStart", >> - (CMPIValue *)&n, CMPI_uint16); >> + (CMPIValue *)&port_num, CMPI_uint16); >> } >> >> if (rule_data.dstportend) { >> - n = atoi(rule_data.dstportend); >> + port_num = atoi(rule_data.dstportend); >> CMSetProperty(inst, "HdrDestPortEnd", >> - (CMPIValue *)&n, CMPI_uint16); >> + (CMPIValue *)&port_num, CMPI_uint16); >> } >> } >> >> @@ -515,7 +517,7 @@ static CMPIInstance *convert_rule_to_instance( >> const char *sys_name = NULL; >> const char *sys_ccname = NULL; >> const char *basename = NULL; >> - int action, direction, priority = 0; >> + uint16_t action, direction, priority = 0; > > This is declared an uint16_t here; however, later on it's stored as a > signed int16: > > priority = convert_priority(rule->priority); > CMSetProperty(inst, "Priority", (CMPIValue *)&priority, > CMPI_sint16); hm ... collateral damage. The "Priority" property is indeed a sint16 (schema/FilterEntry.mof), "Direction" and "Action" are uint16's. > > > Also in convert_filter_to_instance(), there's some similar oddities: > > ... > int direction = 0, priority; oversight ... should be uint16_t direction = 0; int16_t priority; > ... > CMSetProperty(inst, "Direction", (CMPIValue *)&direction, > CMPI_uint16); > > priority = convert_priority(filter->priority); further, we need to change the return type of all the conver_xxx functions ... In my opinion this warrants a V2 patch ... do you prefer a resend of entire series or just this single patch? -- Mit freundlichen Gr??en/Kind Regards Viktor Mihajlovski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martina K?deritz Gesch?ftsf?hrung: Dirk Wittkopp Sitz der Gesellschaft: B?blingen Registergericht: Amtsgericht Stuttgart, HRB 243294 From jferlan at redhat.com Wed Nov 13 16:54:23 2013 From: jferlan at redhat.com (John Ferlan) Date: Wed, 13 Nov 2013 11:54:23 -0500 Subject: [Libvirt-cim] [RESEND PATCH 0/3] libvirt-cim: Fix provider registration In-Reply-To: <1383642213-10260-1-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1383642213-10260-1-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <5283AEBF.9000108@redhat.com> On 11/05/2013 04:03 AM, Viktor Mihajlovski wrote: > The following series fixes two issues in the provider registration > process of the libvirt-cim RPM package. > First, the RPM scriptlets didn't properly reflect the invocation > sequence during an RPM upgrade. This is fixed by the first patch. > Further, the registration using wildcards (*.mof) failed because of > an inter-MOF depedency. Fixed by the second patch. > Finally, the class deletion sequence in provider-register.sh was > prone to failure, resulting in residual entries in the Pegasus > class repository. Addressed by third patch. > > Note: since the redhat.com mailing lists are bouncing my emails > I am sending from a different account temporarily until this is > resolved. > > Viktor Mihajlovski (3): > build: Fix incorrect provider registration in upgrade path > build: Fix provider registration issues > schema: Fix class removal with Pegasus > > libvirt-cim.spec.in | 282 ++++++++++++++++++++++++++++++++++++++++---------- > provider-register.sh | 8 +- > 2 files changed, 231 insertions(+), 59 deletions(-) > I've pushed this upstream John From jferlan at redhat.com Wed Nov 13 16:57:13 2013 From: jferlan at redhat.com (John Ferlan) Date: Wed, 13 Nov 2013 11:57:13 -0500 Subject: [Libvirt-cim] [PATCH 2/4] FilterEntry: Fix endianness issues In-Reply-To: <5283AE08.2030806@linux.vnet.ibm.com> References: <1381492055-11502-1-git-send-email-mihajlov@linux.vnet.ibm.com> <1381492055-11502-3-git-send-email-mihajlov@linux.vnet.ibm.com> <52829D8D.7040805@redhat.com> <5283AE08.2030806@linux.vnet.ibm.com> Message-ID: <5283AF69.4060101@redhat.com> On 11/13/2013 11:51 AM, Viktor Mihajlovski wrote: > > In my opinion this warrants a V2 patch ... do you prefer a > resend of entire series or just this single patch? > Just this patch - I will push 1, 2, & 4 shortly... John From mihajlov at linux.vnet.ibm.com Wed Nov 13 18:07:10 2013 From: mihajlov at linux.vnet.ibm.com (Viktor Mihajlovski) Date: Wed, 13 Nov 2013 19:07:10 +0100 Subject: [Libvirt-cim] [PATCHv2 2/4] FilterEntry: Fix endianness issues In-Reply-To: <5283AF69.4060101@redhat.com> References: <5283AF69.4060101@redhat.com> Message-ID: <1384366030-11842-1-git-send-email-mihajlov@linux.vnet.ibm.com> From: Thilo Boehm A number of CIM properties was set in an endianness-unsafe manner leading to failures on big endian systems. Signed-off-by: Thilo Boehm Signed-off-by: Viktor Mihajlovski --- src/Virt_FilterEntry.c | 53 +++++++++++++++++++++++++----------------------- src/Virt_FilterEntry.h | 2 +- src/Virt_FilterList.c | 3 ++- 3 files changed, 31 insertions(+), 27 deletions(-) V2 Changes: - The FilterEntry "Priority" property is a signed 16bit value, fix the variable declaration accordingly - Change the convert_xxx functions to return a properly typed int value diff --git a/src/Virt_FilterEntry.c b/src/Virt_FilterEntry.c index b7042da..ab4a512 100644 --- a/src/Virt_FilterEntry.c +++ b/src/Virt_FilterEntry.c @@ -59,8 +59,8 @@ struct rule_data_t { const char *dstportend; }; -static int octets_from_mac(const char * s, unsigned int *buffer, - unsigned int size) +static int octets_from_mac(const char * s, uint8_t *buffer, + unsigned int size) { unsigned int _buffer[6]; unsigned int i, n = 0; @@ -86,8 +86,8 @@ static int octets_from_mac(const char * s, unsigned int *buffer, return n; } -static int octets_from_ip(const char * s, unsigned int *buffer, - unsigned int size) +static int octets_from_ip(const char * s, uint8_t *buffer, + unsigned int size) { struct in6_addr addr; unsigned int family = 0; @@ -116,7 +116,8 @@ static int octets_from_ip(const char * s, unsigned int *buffer, return n; } -static CMPIArray *octets_to_cmpi(const CMPIBroker *broker, unsigned int *bytes, int size) +static CMPIArray *octets_to_cmpi(const CMPIBroker *broker, uint8_t *bytes, + int size) { CMPIStatus s = {CMPI_RC_OK, NULL}; CMPIArray *array = NULL; @@ -173,7 +174,7 @@ static char *cidr_to_str(const char *cidr) return ret; } -static int convert_direction(const char *s) +static uint16_t convert_direction(const char *s) { enum {NOT_APPLICABLE, INPUT, OUTPUT, BOTH} direction = NOT_APPLICABLE; @@ -189,7 +190,7 @@ static int convert_direction(const char *s) return direction; } -int convert_priority(const char *s) +int16_t convert_priority(const char *s) { if (s == NULL) return 0; @@ -197,7 +198,7 @@ int convert_priority(const char *s) return atoi(s); } -static int convert_action(const char *s) +static uint16_t convert_action(const char *s) { enum {NONE=0, ACCEPT, DENY, REJECT, RETURN, CONTINUE} action = NONE; @@ -216,7 +217,7 @@ static int convert_action(const char *s) return action; } -static unsigned long convert_protocol_id(const char *s) +static uint16_t convert_protocol_id(const char *s) { enum {NONE = 0, IPV4 = 2048, ARP = 2054, RARP = 32821, IPV6 = 34525} id = NONE; @@ -239,7 +240,7 @@ static void convert_mac_rule_to_instance( CMPIInstance *inst, const CMPIBroker *broker) { - unsigned int bytes[48]; + uint8_t bytes[48]; unsigned int size = 0; CMPIArray *array = NULL; @@ -280,7 +281,7 @@ static void convert_mac_rule_to_instance( (CMPIValue *)&array, CMPI_uint8A); if (rule->var.mac.protocol_id != NULL) { - unsigned long n = convert_protocol_id(rule->var.mac.protocol_id); + uint16_t n = convert_protocol_id(rule->var.mac.protocol_id); /* Unknown protocolid string. Try converting from hexadecimal value */ if (n == 0) @@ -366,18 +367,19 @@ static void convert_ip_rule_to_instance( CMPIInstance *inst, const CMPIBroker *broker) { - unsigned int bytes[48]; + uint8_t bytes[48]; unsigned int size = 0; - unsigned int n = 0; + uint8_t ipver_num = 0; + uint16_t port_num = 0; CMPIArray *array = NULL; struct rule_data_t rule_data; if (strstr(rule->protocol_id, "v6")) - n = 6; + ipver_num = 6; else - n = 4; + ipver_num = 4; - CMSetProperty(inst, "HdrIPVersion",(CMPIValue *)&n, CMPI_uint8); + CMSetProperty(inst, "HdrIPVersion",(CMPIValue *)&ipver_num, CMPI_uint8); fill_rule_data(rule, &rule_data); @@ -480,27 +482,27 @@ static void convert_ip_rule_to_instance( } if (rule_data.srcportstart) { - n = atoi(rule_data.srcportstart); + port_num = atoi(rule_data.srcportstart); CMSetProperty(inst, "HdrSrcPortStart", - (CMPIValue *)&n, CMPI_uint16); + (CMPIValue *)&port_num, CMPI_uint16); } if (rule_data.srcportend) { - n = atoi(rule_data.srcportend); + port_num = atoi(rule_data.srcportend); CMSetProperty(inst, "HdrSrcPortEnd", - (CMPIValue *)&n, CMPI_uint16); + (CMPIValue *)&port_num, CMPI_uint16); } if (rule_data.dstportstart) { - n = atoi(rule_data.dstportstart); + port_num = atoi(rule_data.dstportstart); CMSetProperty(inst, "HdrDestPortStart", - (CMPIValue *)&n, CMPI_uint16); + (CMPIValue *)&port_num, CMPI_uint16); } if (rule_data.dstportend) { - n = atoi(rule_data.dstportend); + port_num = atoi(rule_data.dstportend); CMSetProperty(inst, "HdrDestPortEnd", - (CMPIValue *)&n, CMPI_uint16); + (CMPIValue *)&port_num, CMPI_uint16); } } @@ -515,7 +517,8 @@ static CMPIInstance *convert_rule_to_instance( const char *sys_name = NULL; const char *sys_ccname = NULL; const char *basename = NULL; - int action, direction, priority = 0; + uint16_t action, direction; + int16_t priority; void (*convert_f)(struct acl_rule*, CMPIInstance*, const CMPIBroker*); diff --git a/src/Virt_FilterEntry.h b/src/Virt_FilterEntry.h index 5057fb0..0589aa0 100644 --- a/src/Virt_FilterEntry.h +++ b/src/Virt_FilterEntry.h @@ -77,7 +77,7 @@ CMPIStatus instance_from_rule( * * @param s A pointer to a string representing the priority */ -int convert_priority(const char *s); +int16_t convert_priority(const char *s); #endif /* diff --git a/src/Virt_FilterList.c b/src/Virt_FilterList.c index 7026d8b..b248004 100644 --- a/src/Virt_FilterList.c +++ b/src/Virt_FilterList.c @@ -45,7 +45,8 @@ static CMPIInstance *convert_filter_to_instance( CMPIInstance *inst = NULL; const char *sys_name = NULL; const char *sys_ccname = NULL; - int direction = 0, priority; + uint16_t direction = 0; + int16_t priority; inst = get_typed_instance(broker, CLASSNAME(reference), -- 1.7.9.5 From jferlan at redhat.com Wed Nov 13 18:34:19 2013 From: jferlan at redhat.com (John Ferlan) Date: Wed, 13 Nov 2013 13:34:19 -0500 Subject: [Libvirt-cim] [PATCHv2 2/4] FilterEntry: Fix endianness issues In-Reply-To: <1384366030-11842-1-git-send-email-mihajlov@linux.vnet.ibm.com> References: <5283AF69.4060101@redhat.com> <1384366030-11842-1-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <5283C62B.1040606@redhat.com> On 11/13/2013 01:07 PM, Viktor Mihajlovski wrote: > From: Thilo Boehm > > A number of CIM properties was set in an endianness-unsafe manner > leading to failures on big endian systems. > > Signed-off-by: Thilo Boehm > Signed-off-by: Viktor Mihajlovski > --- > src/Virt_FilterEntry.c | 53 +++++++++++++++++++++++++----------------------- > src/Virt_FilterEntry.h | 2 +- > src/Virt_FilterList.c | 3 ++- > 3 files changed, 31 insertions(+), 27 deletions(-) > > V2 Changes: > - The FilterEntry "Priority" property is a signed 16bit value, fix > the variable declaration accordingly > - Change the convert_xxx functions to return a properly typed int value > ACK and pushed too John From jferlan at redhat.com Wed Nov 13 23:08:16 2013 From: jferlan at redhat.com (John Ferlan) Date: Wed, 13 Nov 2013 18:08:16 -0500 Subject: [Libvirt-cim] [PATCH V2 01/48] Add others member for saving unsupported tag and unknown device In-Reply-To: <5282F1A8.3@linux.vnet.ibm.com> References: <1382928377-16331-1-git-send-email-gesaint@linux.vnet.ibm.com> <1382928377-16331-2-git-send-email-gesaint@linux.vnet.ibm.com> <5282CEDF.4090506@redhat.com> <5282F1A8.3@linux.vnet.ibm.com> Message-ID: <52840660.5030400@redhat.com> On 11/12/2013 10:27 PM, Xu Wang wrote: > > ? 2013/11/13 8:59, John Ferlan ??: > Dear John, > My idea is, using a data structure (a link list) to keep the elements > left after all kinds of > virtual devices fetched tags they needed from xml. And if I just mark a > node of others with > status instead of delete it, the whole structure of xml could be saved > as well. > The introduction of 'id' in 'others' structure is to identify several > with the same > name. The console device support new introduced and some tags in > 'unknown_device' > have to use 'id' to identify elements. It maybe a little coarse now so I > need any suggestion > from you to make it works better. If you have any better idea to solve > the unsupported > tags missing issue please share with me :-) Because so many patches > about it merged > and there will be more in the future. > These 48 patches just the first part in my designing. After that I want > to build the > management about all data (even in 'others' member). And all of these > updates are > compatitable with older version libvirt-cim. So the upper layer need > little change and > keep the original feature but could make libvirt-cim becomes more powerful. > XFAIL about hotplug disappeared after updated. About the warning you > mentioned above > I'll check and fix it in the new version. > > Thanks, > Xu Wang I don't have any thoughts on better ideas - it's just not my focus. There's a lot of changes being made in this series which I suppose in hindsight would have been nice to have been broken up a bit more to make it manageable to review, understand, and test. I've started with the first 3 patches as it seems all they do is create infrastructure and "for the most part" no one should be calling them. I started with patch 1... got to patch 2 and had issues. I ran patch 2 through coverity and it complained about the memset() in cleanup_virt_device(). A recent patch changed that from "memset(&dev->dev, 0, sizeof(dev->dev));" to "memset(dev, 0, sizeof(*dev));". Coverity complained about the path from "classify_resources()" for CIM_RES_TYPE_DISK into the rasd_to_vdev() call, then into _sysvirt_rasd_to_vdev(), disk_rasd_to_vdev(), and finally cleanup_virt_device(). It seems the change to device_parsing.h, inclusion by Virt_VirtualSystemManagementService.c, and usage of virt_device as a static structure caused the "issue" as a "clean" build fixed things. Although it took a while for me to figure it out. I suppose that's more a build/Makefile issue than anything else... The second issue I've run into is that by only applying the first two patches, I can get a test to cause cimprovagt to fail: -------------------------------------------------------------------- VirtualSystemManagementService - 16_removeresource.py: FAIL ERROR - (1, u'CIM_ERR_FAILED: Lost connection with cimprovagt "libvirt-cim".') InvokeMethod(RemoveResourceSettings): CIM_ERR_FAILED: Lost connection with cimprovagt "libvirt-cim". -------------------------------------------------------------------- This one's really frustrating as I'm really perplexed why a possible fix could work. It seems if I don't call cleanup_others() from cleanup_unknown_device(), then I don't get that core. I have no idea why though. Is it possibly related to the static usage of virt_device? I see this even after a clean build. Maybe this "cleanup_unknown_device()" and it's counter-part "dev_unknown" should have been added after the others work. It seems there are two things being done at once - creating an 'others' list to store the read xml and creating a list of unknown devices. You've done a lot of work and it seems promising as a way handle new tags. Of course I ask myself why is it so important to go through this exercise. Are the libvirt API's deficient? I would think libvirt should be the only thing handling XML format and the applications that use libvirt would have API's to create, define, get, set, start, stop, etc VM's. Having to store the XML is "dangerous" insomuch as we've already found it can change and libvirt-cim cannot keep up. Anyway, as an exercise for me to learn and because I pushed a number of changes which conflict with what you've already done... I will try to help with the effort to compact each patch series into manageable parts. FWIW: I pushed the other patches because they were posted first - I just went in order. John From cngesaint at gmail.com Thu Nov 14 06:25:47 2013 From: cngesaint at gmail.com (Xu Wang) Date: Thu, 14 Nov 2013 14:25:47 +0800 Subject: [Libvirt-cim] [PATCH V2 01/48] Add others member for saving unsupported tag and unknown device In-Reply-To: <52840660.5030400@redhat.com> References: <1382928377-16331-1-git-send-email-gesaint@linux.vnet.ibm.com> <1382928377-16331-2-git-send-email-gesaint@linux.vnet.ibm.com> <5282CEDF.4090506@redhat.com> <5282F1A8.3@linux.vnet.ibm.com> <52840660.5030400@redhat.com> Message-ID: <52846CEB.8040308@gmail.com> Since I started to work on libvirt-cim almost every bug is about new tags support. My first patch is about in the . The second one is in the . And other one coming soon is in , and endless like this. So I got a mission that is designing/implementing a long term solution to solve this category of bug. So I have a dream, even re-write all content of libxutil/device_parsing.c and libxutil/xmlgen.c to reach that goal. I'll continue to work on the issues you found. And update the design unsuitable. I'll make a new version after finished. Just rebasing work is a little boring. Thank you for taking so much time to review and test them. Thanks, Xu Wang ? 2013/11/14 7:08, John Ferlan ??: > On 11/12/2013 10:27 PM, Xu Wang wrote: >> ? 2013/11/13 8:59, John Ferlan ??: >> Dear John, >> My idea is, using a data structure (a link list) to keep the elements >> left after all kinds of >> virtual devices fetched tags they needed from xml. And if I just mark a >> node of others with >> status instead of delete it, the whole structure of xml could be saved >> as well. >> The introduction of 'id' in 'others' structure is to identify several >> with the same >> name. The console device support new introduced and some tags in >> 'unknown_device' >> have to use 'id' to identify elements. It maybe a little coarse now so I >> need any suggestion >> from you to make it works better. If you have any better idea to solve >> the unsupported >> tags missing issue please share with me :-) Because so many patches >> about it merged >> and there will be more in the future. >> These 48 patches just the first part in my designing. After that I want >> to build the >> management about all data (even in 'others' member). And all of these >> updates are >> compatitable with older version libvirt-cim. So the upper layer need >> little change and >> keep the original feature but could make libvirt-cim becomes more powerful. >> XFAIL about hotplug disappeared after updated. About the warning you >> mentioned above >> I'll check and fix it in the new version. >> >> Thanks, >> Xu Wang > I don't have any thoughts on better ideas - it's just not my focus. > There's a lot of changes being made in this series which I suppose in > hindsight would have been nice to have been broken up a bit more to make > it manageable to review, understand, and test. I've started with the > first 3 patches as it seems all they do is create infrastructure and > "for the most part" no one should be calling them. > > I started with patch 1... got to patch 2 and had issues. I ran patch 2 > through coverity and it complained about the memset() in > cleanup_virt_device(). > > A recent patch changed that from "memset(&dev->dev, 0, > sizeof(dev->dev));" to "memset(dev, 0, sizeof(*dev));". Coverity > complained about the path from "classify_resources()" for > CIM_RES_TYPE_DISK into the rasd_to_vdev() call, then into > _sysvirt_rasd_to_vdev(), disk_rasd_to_vdev(), and finally > cleanup_virt_device(). > > It seems the change to device_parsing.h, inclusion by > Virt_VirtualSystemManagementService.c, and usage of virt_device as a > static structure caused the "issue" as a "clean" build fixed things. > Although it took a while for me to figure it out. I suppose that's more > a build/Makefile issue than anything else... > > The second issue I've run into is that by only applying the first two > patches, I can get a test to cause cimprovagt to fail: > > -------------------------------------------------------------------- > VirtualSystemManagementService - 16_removeresource.py: FAIL > ERROR - (1, u'CIM_ERR_FAILED: Lost connection with cimprovagt > "libvirt-cim".') > InvokeMethod(RemoveResourceSettings): CIM_ERR_FAILED: Lost connection > with cimprovagt "libvirt-cim". > -------------------------------------------------------------------- > > This one's really frustrating as I'm really perplexed why a possible fix > could work. It seems if I don't call cleanup_others() from > cleanup_unknown_device(), then I don't get that core. I have no idea > why though. Is it possibly related to the static usage of virt_device? I > see this even after a clean build. > > Maybe this "cleanup_unknown_device()" and it's counter-part > "dev_unknown" should have been added after the others work. It seems > there are two things being done at once - creating an 'others' list to > store the read xml and creating a list of unknown devices. > > You've done a lot of work and it seems promising as a way handle new > tags. Of course I ask myself why is it so important to go through this > exercise. Are the libvirt API's deficient? I would think libvirt > should be the only thing handling XML format and the applications that > use libvirt would have API's to create, define, get, set, start, stop, > etc VM's. Having to store the XML is "dangerous" insomuch as we've > already found it can change and libvirt-cim cannot keep up. > > Anyway, as an exercise for me to learn and because I pushed a number of > changes which conflict with what you've already done... I will try to > help with the effort to compact each patch series into manageable parts. > FWIW: I pushed the other patches because they were posted first - I just > went in order. > > John From daniel.hansel at linux.vnet.ibm.com Thu Nov 14 10:28:16 2013 From: daniel.hansel at linux.vnet.ibm.com (Daniel Hansel) Date: Thu, 14 Nov 2013 11:28:16 +0100 Subject: [Libvirt-cim] [RESEND PATCH 0/3] libvirt-cim: Fix provider registration In-Reply-To: <5282668C.8080000@redhat.com> References: <1383642213-10260-1-git-send-email-mihajlov@linux.vnet.ibm.com> <528144F6.6030907@redhat.com> <52825ACA.4000704@de.ibm.com> <5282668C.8080000@redhat.com> Message-ID: <5284A5C0.8090101@linux.vnet.ibm.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 12.11.2013 18:34, John Ferlan wrote: > >>> >>> I'm much of an expert in these matters - in fact pre-novice would summarize my experience with RPM files :-) >>> >>> Things seem reasonable; however, I'm in a bit of a quandary now as I cannot get my libvirt-cim environment to work in order to test. This past Friday I did do a yum update and since that time >>> I cannot seem to get the libvirt-cim provider and cimserver to talk - quite frustrating. >>> >>> I was actually hoping to spend this week reviewing and testing libvirt-cim patches... Now I'm just trying to figure out why two best friends won't speak to each other any more :-) >> Hi John, >> >> could describe your libvirt-cim environment a bit more detailed (e.g. OS version, installed package versions of libvirt and libvirt-cim, cimserver, etc.)? >> >> Maybe we could figure out the error together. >> > > > > Sure - I am running f19 on a lenovo t530 - it's my work supplied laptop > > I generally use the top of the libvirt and libvirt-cim tree, although those didn't change when this issue was discovered. > > Prior to 10/30/13, I'm not sure which version of tog-pegasus* was installed; however, as of that day the following was installed: > > Oct 30 08:00:41 Updated: 2:tog-pegasus-libs-2.12.1-8.fc19.x86_64 Oct 30 08:01:08 Updated: 2:tog-pegasus-2.12.1-8.fc19.x86_64 Oct 30 08:01:39 Updated: 2:tog-pegasus-devel-2.12.1-8.fc19.x86_64 > > > a 'cimprovider -l' does not return any KVM_ modules a sure sign of things not working... > > In order to help dig, I enabled debugging: > > cimconfig -s traceLevel=4 -c cimconfig -s traceComponents=ALL -c > > Looking at the cimserver.trc file I find these (sorry about the formatting my auto line wrap is on): > > 1384276573s-184895us: Repository [7944:139824043710528:FileBasedStore.cpp:631]: Namespace: root#PG_InterOp ignored -- subdirectories are not correctly formed 1384276573s-186825us: L10N > [7944:139824043710528:MessageLoader.cpp:418]: Message ID = Common.InternalException.CANNOT_OPEN_DIRECTORY 1384276573s-187030us: Repository [7944:139824043710528:FileBasedStore.cpp:1347]: > Namespace: root#PG_InterOp ignored -- subdirectories are not correctly formed 1384276573s-194463us: ProviderManager [7944:139824043710528:ProviderRegistrationManager.cpp:1934]: nameSpace = > root/interop; className = PG_ProviderModule > > I think you get the picture though - there's something about PG_InterOp which is different. Since it's one of those things that 2.12.1-8 release notes discusses as changing in 2.12.1-4, I have a > feeling whatever it is the libvirt-cim.spec does in order to "link up" as a provider, does not work in the same way any more. > Hi John, after investigating in this problem we have seen that with tog-pegasus version 2.12.1-5 the namespace for base classes was changed from root/PG_Interop to root/interop. tog-pegasus can be build using an option that changes this namespace to one of these 2 namespaces. As far as we could see RedHat is using this option (and the new namespace) in their build of tog-pegasus. The cim providers that should be installed have to use the correct namespace to find the right classes. This would lead to an incompatibility of cim providers AND clients. Therefore the pegasus community does NOT recommend the change to the new namespace until a backwards compatible way is implemented in tog-pegasus. Kind regards, Daniel > > Doing a : > > wbemcli ein http://root:password at localhost:5988/root/virt:KVM_VirtualSystemManagementService > > gets: > > 1384277180s-587013us: ProviderManager [9934:140222156466240:ProviderRegistrationManager.cpp:395]: nameSpace = root/virt; className = KVM_VirtualSystemManagementService; className = > root/virtkvm_virtualsystemmanagementserviceinstance 1384277180s-587020us: ProviderManager [9934:140222156466240:ProviderRegistrationManager.cpp:406]: Provider capability has not been registered > yet. 1384277180s-587028us: Dispatcher [9934:140222156466240:CIMOperationRequestDispatcher.cpp:1465]: Provider for KVM_VirtualSystemManagementService not found. > > But no output from the wbemcli command. > > I did try "going backwards" and installing an older rpm, but was not successful. So I borrowed another RHEL6.5 system and am testing over there today. > > > John > > - -- Mit freundlichen Gr??en / Kind regards Daniel Hansel IBM Deutschland Research & Development GmbH Vorsitzende des Aufsichtsrats: Martina Koederitz Gesch?ftsf?hrung: Dirk Wittkopp Sitz der Gesellschaft: B?blingen Registergericht: Amtsgericht Stuttgart, HRB 243294 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJShKXAAAoJEJszBMQcau4WLEEP/juXbDAah/EcsHahjBUT5PN6 oNnzVK0PJrbM5JAFszc7gp4N5gyxV50IVIexD8s3pqOvob7BlBYgIRuw3uzW9jcP UxMwsYz4StYPpGws4IjamisIHIeAwt3JfQapGdi5VJ0YJDRfsMacGdQbB8pmYbWT aiNQeQu6lu3i+KEBMbWzp9N210jMQwJJrG55I3VmxlbJsZpbHvTdRXXp4waQqArK sIHZgSRV0WY+pdG6QQuQ/I5lJQVjLuVUXd9wu/rX19Ii9S6U4Gequ+44/TIacsRV dAJEGpCrpuydl+RJn24RXACH5FwrnpKoVsdp23KpY+faq+Z1ICCLAUcpzVvgeUjl YB7I9HTrmvpQ16X6tDzPiNP7sZD03sZgnT5yqat8ZN9tb9Uju8ZVKLJtcSQfZ1LA M1qTwlwFtUSY/qFkBD07s5TLEDPfLucJWrHg5sNOUs1gB9nHW9rv9eUuIwSl114h doWwUhj1TtyfYu/SQFIjm7Y63UYr7i7+g7QlzL/jvKMpI0kUgZfU+m7bhSVFcVm0 DGZ14u2GXOQKam2YnsSb8DiUiOSO0y7qmNCh6bYhEuhxussdB8Ls7xdtN6lKF+Ej hAW3LRIdO3QgAj0vIOswFg9K46+EMvSIJglhah0poG1+xZ13zNxLTM3EOUYWS3Wm vzspVoxGLLagzog0ywy/ =TF+o -----END PGP SIGNATURE----- From mihajlov at linux.vnet.ibm.com Thu Nov 14 16:59:58 2013 From: mihajlov at linux.vnet.ibm.com (Viktor Mihajlovski) Date: Thu, 14 Nov 2013 17:59:58 +0100 Subject: [Libvirt-cim] [RESEND PATCH 0/3] libvirt-cim: Fix provider registration In-Reply-To: <5283AEBF.9000108@redhat.com> References: <1383642213-10260-1-git-send-email-mihajlov@linux.vnet.ibm.com> <5283AEBF.9000108@redhat.com> Message-ID: <5285018E.9040901@linux.vnet.ibm.com> On 11/13/2013 05:54 PM, John Ferlan wrote: > > > I've pushed this upstream > > John thanks for reviewing (and pushing) all the patches -- Mit freundlichen Gr??en/Kind Regards Viktor Mihajlovski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martina K?deritz Gesch?ftsf?hrung: Dirk Wittkopp Sitz der Gesellschaft: B?blingen Registergericht: Amtsgericht Stuttgart, HRB 243294 From jferlan at redhat.com Fri Nov 15 00:23:50 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:23:50 -0500 Subject: [Libvirt-cim] [PATCH 01/20] Add others member for saving unsupported tag and unknown device In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-2-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.h | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/libxkutil/device_parsing.h b/libxkutil/device_parsing.h index 92427c1..603f29a 100644 --- a/libxkutil/device_parsing.h +++ b/libxkutil/device_parsing.h @@ -39,6 +39,27 @@ struct device_address { char **value; }; +/* The structure for saving unknown tag in the xml */ +enum others_type { + TYPE_PROP, + TYPE_NODE +}; + +struct others { + /* To identify the different tags with same name */ + int id; + char *name; + int parent_id; + char *parent_name; + enum others_type type; + char *value; + struct others *next; + enum { + ACTIVE, + INACTIVE + } status; +}; + struct vsi_device { char *vsi_type; char *manager_id; -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:23:51 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:23:51 -0500 Subject: [Libvirt-cim] [PATCH 02/20] Add others and unknown_device clean up In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-3-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c index 56e39c7..6cdcfa9 100644 --- a/libxkutil/device_parsing.c +++ b/libxkutil/device_parsing.c @@ -79,6 +79,28 @@ static void cleanup_device_address(struct device_address *addr) addr->ct = 0; } +static void cleanup_node_of_others(struct others *others) +{ + if (others == NULL) + return; + + free(others->name); + free(others->parent_name); + free(others->value); + free(others); +} + +static void cleanup_others(struct others *others) +{ + struct others *head = others; + + while (others) { + head = others->next; + cleanup_node_of_others(others); + others = head; + } +} + static void cleanup_disk_device(struct disk_device *dev) { if (dev == NULL) -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:23:49 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:23:49 -0500 Subject: [Libvirt-cim] [PATCH 00/20] REWORK/PARIAL: Changes to solve unsupported tag issue Message-ID: <1384475049-11435-1-git-send-email-jferlan@redhat.com> This is a *partial rework* of Xu Wang's patches sent last month: https://www.redhat.com/archives/libvirt-cim/2013-October/msg00081.html Although not the complete set of changes - it's a good stopping point insomuch as it handles the "others" parsing. If this looks good, I can push it, then work through the changes to write the xml. I have run all the changes through cimtest - even with the patches on the list from Viktor. No new issues are found. Changes to the original patches 1. I rebased to top of tree as of today (11/14/13) 2. I reworked each of the patches to only add the 'others' to the particular *_device structure. This was done to be able to show the progression and to ensure I didn't forget something 2a. This found that there needed to be a cleanup_vcpu_device() and cleanup_mem_device() since both added "others" to their structure but were never cleaned up during cleanup_virt_device() 2b. I did not see a need for "others" in "vnc_device" and "sdl_device" 2c. I added a cleanup_others() call in _get_dominfo() 3. I added fetch_device_address_from_others() to replace parse_device_address(). This bridges the "gap" bewteen the parsing of the
tag that was recently added and the 'others' parsing code which didn't handle it. All I did was traverse the others list looking for parent name and id match for TYPE_PROP entries. Essentially anything inside of us then copied in to name/value structure managed by devaddr. Once the last caller of parse_device_address() was removed, that function went away 4. I changed the type for unknown from CIM_RES_TYPE_UNKNOWN to CIM_RES_TYPE_UNKDEV. Turns out the former is used in other contexts and if/when cleanup_virt_device() was called from one of those contexts the result was a core in cimprovagt. This was seen in the cimtest for VirtualSystemManagementService 16_removeresource.py. 4. Various "bug fixes" based on a coverity run and what I saw in code review from Boris. - Original code had a "ddev->disk_type == DISK_PHY;" - There was a "if (node->name == NULL)" check in parse_graphics_device after "else if (STREQC(gdev->type, "pty")) {" which coverity flagged as unnecessary since earlier code assumed node->name != NULL - In parse_os() there was a "STREQC(dominfo->os_info.pv.type, "linux")" which needed a "dominfo->os_info.pv.type &&" prior to it since the value could have been NULL according to a check earlier in the routine - I added the "free(dev->name);" to cleanup_unknown_device() - Checks for others->{name|parent_name|value} were removed. Since the structures are calloc()'d and free(NULL) does nothing, this is OK. Xu Wang (20): Add others member for saving unsupported tag and unknown device Add others and unknown_device clean up Add basic operations for reading data from xml node Fix xml parsing algorithm for parse_fs_device() Fix xml parsing algorithm for parse_block_device() Fix xml parsing algorithm for parse_vsi_device() Fix xml parsing algorithm for parse_net_device() Fix xml parsing algorithm for parse_vcpu_device() Fix xml parsing algorithm for parse_emu_device() Fix xml parsing algorithm for parse_mem_device() Fix xml parsing algorithm for parse_console_device() Fix xml parsing algorithm for parse_graphics_device() Fix xml parsing algorithm for parse_input_device() Add parse_unknown_device() Add parse_devices() for unknown type in get_dominfo_from_xml() Fix xml parsing algorithm in parse_domain() Fix xml parsing algorithm in parse_os() Fix xml parsing algorithm in parse_features() Add dup function for device copy Add type CIM_RES_TYPE_DELETED and modify type as it after resource_del libxkutil/device_parsing.c | 2055 ++++++++++++++++++++++++----- libxkutil/device_parsing.h | 58 + src/Virt_VirtualSystemManagementService.c | 2 +- src/svpc_types.h | 2 + 4 files changed, 1774 insertions(+), 343 deletions(-) -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:23:53 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:23:53 -0500 Subject: [Libvirt-cim] [PATCH 04/20] Fix xml parsing algorithm for parse_fs_device() In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-5-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.c | 156 ++++++++++++++++++++++++++++++++++++++------- libxkutil/device_parsing.h | 1 + 2 files changed, 134 insertions(+), 23 deletions(-) diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c index 610712f..20b35e1 100644 --- a/libxkutil/device_parsing.c +++ b/libxkutil/device_parsing.c @@ -116,6 +116,7 @@ static void cleanup_disk_device(struct disk_device *dev) free(dev->bus_type); free(dev->access_mode); cleanup_device_address(&dev->address); + cleanup_others(dev->others); } static void cleanup_vsi_device(struct vsi_device *dev) @@ -453,6 +454,45 @@ static int parse_device_address(xmlNode *anode, struct device_address *devaddr) } /* + * Device addresses can have many forms based on the 'type' field, so rather + * than try to create a function that could parse each particular option, we + * will just fetch all the name/value pairs with "address" as the parent name + * and store them in a devaddr array to be parsed elsewhere. + * + * This function will also assume that the 'id' and 'parentid' values are -1 + * at least for now + */ +static int fetch_device_address_from_others(struct others **head, + struct device_address *devaddr) +{ + struct others *tmp = *head; + + while (tmp) { + if (compare_param_int(tmp->id, -1) && + compare_param_int(tmp->parent_id, -1) && + compare_param_str(tmp->parent_name, "address") && + tmp->type == TYPE_PROP && + tmp->status == ACTIVE) { + + if (!add_device_address_property(devaddr, tmp->name, + tmp->value)) + goto err; + + /* Since we're now managing it... */ + tmp->status = INACTIVE; + } + tmp = tmp->next; + } + + return 1; + + err: + cleanup_device_address(devaddr); + + return 0; +} + +/* * This function is just used for debugging. If you found something wrong * please call this function and check the result of output in the logs. */ @@ -753,7 +793,8 @@ static int parse_fs_device(xmlNode *dnode, struct virt_device **vdevs) { struct virt_device *vdev = NULL; struct disk_device *ddev = NULL; - xmlNode *child = NULL; + + CU_DEBUG("Enter parse_fs_device()."); vdev = calloc(1, sizeof(*vdev)); if (vdev == NULL) @@ -761,37 +802,106 @@ static int parse_fs_device(xmlNode *dnode, struct virt_device **vdevs) ddev = (&vdev->dev.disk); - ddev->type = get_attr_value(dnode, "type"); + ddev->others = parse_data_to_others(ddev->others, + dnode, + 0, + BAD_CAST "devices"); + if (ddev->others == NULL) { + CU_DEBUG("parse xml failed."); + goto err; + } + + /* fetch out tag from others. It will be removed + * after others management finished. */ + fetch_from_others(&ddev->others, + -1, + "filesystem", + TYPE_NODE, + -1, + "devices"); + + ddev->type = fetch_from_others(&ddev->others, + -1, + "type", + TYPE_PROP, + -1, + (char *)dnode->name); if (ddev->type == NULL) { CU_DEBUG("No type"); goto err; } - ddev->access_mode = get_attr_value(dnode, "accessmode"); + ddev->access_mode = fetch_from_others(&ddev->others, + -1, + "accessmode", + TYPE_PROP, + -1, + (char *)dnode->name); + + if (seek_in_others(&ddev->others, + -1, + "source", + TYPE_NODE, + -1, + (char *)dnode->name)) { + ddev->source = fetch_from_others(&ddev->others, + -1, + "dir", + TYPE_PROP, + -1, + "source"); + + + if (ddev->source == NULL) { + CU_DEBUG("no source dir"); + goto err; + } + } - for (child = dnode->children; child != NULL; child = child->next) { - if (XSTREQ(child->name, "source")) { - ddev->source = get_attr_value(child, "dir"); - if (ddev->source == NULL) { - CU_DEBUG("No source dir"); - goto err; - } - } else if (XSTREQ(child->name, "target")) { - ddev->virtual_dev = get_attr_value(child, "dir"); - if (ddev->virtual_dev == NULL) { - CU_DEBUG("No target dir"); - goto err; - } - } else if (XSTREQ(child->name, "driver")) { - ddev->driver_type = get_attr_value(child, "type"); - } else if (XSTREQ(child->name, "address")) { - parse_device_address(child, &ddev->address); + if (seek_in_others(&ddev->others, + -1, + "target", + TYPE_NODE, + -1, + (char *)dnode->name)) { + ddev->virtual_dev = fetch_from_others(&ddev->others, + -1, + "dir", + TYPE_PROP, + -1, + "target"); + + if (ddev->virtual_dev == NULL) { + CU_DEBUG("no target dir"); + goto err; } } - if ((ddev->source == NULL) || (ddev->virtual_dev == NULL)) { - CU_DEBUG("S: %s D: %s", ddev->source, ddev->virtual_dev); - goto err; + if (seek_in_others(&ddev->others, + -1, + "driver", + TYPE_NODE, + -1, + (char *)dnode->name)) { + ddev->driver_type = fetch_from_others(&ddev->others, + -1, + "type", + TYPE_PROP, + -1, + "driver"); + } + + if (seek_in_others(&ddev->others, + -1, + "address", + TYPE_NODE, + -1, + (char *)dnode->name)) { + if (!fetch_device_address_from_others(&ddev->others, + &ddev->address)) { + CU_DEBUG("error fetching device address"); + goto err; + } } ddev->disk_type = DISK_FS; diff --git a/libxkutil/device_parsing.h b/libxkutil/device_parsing.h index 92d8638..6e8fe25 100644 --- a/libxkutil/device_parsing.h +++ b/libxkutil/device_parsing.h @@ -84,6 +84,7 @@ struct disk_device { char *cache; char *access_mode; /* access modes for DISK_FS (filesystem) type */ struct device_address address; + struct others *others; }; struct net_device { -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:23:52 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:23:52 -0500 Subject: [Libvirt-cim] [PATCH 03/20] Add basic operations for reading data from xml node In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-4-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.c | 297 +++++++++++++++++++++++++++++++++++++++++++++ libxkutil/device_parsing.h | 17 +++ 2 files changed, 314 insertions(+) diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c index 6cdcfa9..610712f 100644 --- a/libxkutil/device_parsing.c +++ b/libxkutil/device_parsing.c @@ -452,6 +452,303 @@ static int parse_device_address(xmlNode *anode, struct device_address *devaddr) return 0; } +/* + * This function is just used for debugging. If you found something wrong + * please call this function and check the result of output in the logs. + */ +void print_others(struct others *head) +{ + CU_DEBUG("**************************************************"); + while (head) { + CU_DEBUG("---------------------------"); + CU_DEBUG("- others id: %d", head->id); + CU_DEBUG("- others name: %s", head->name); + CU_DEBUG("- others value: %s", head->value); + CU_DEBUG("- others type: %d", head->type); + CU_DEBUG("- others parent_id: %d", head->parent_id); + CU_DEBUG("- others parent_name: %s", head->parent_name); + if (head->status == ACTIVE) { + CU_DEBUG("- others status: ACTIVE"); + } else { + CU_DEBUG("- others status: INACTIVE"); + } + CU_DEBUG("---------------------------"); + head = head->next; + } + CU_DEBUG("**************************************************"); +} + +/* + * Add one node into head link list. All items are set as parameters. + * + * @head: The link list head pointer to be added. + * @node: Where the value of node come from. + * @name: Name of this link list node (maybe a node or an attribute in xml). + * @type: TYPE_NODE: Stands for the value from a xml node. + * TYPE_PROP: Means that this value is an attribute of a node. + * @parent_id: The id of parent node. + * @parent_name: Name of parent node. + */ +static struct others *add_others(struct others *head, + xmlNode *node, + const xmlChar *name, + enum others_type type, + int parent_id, + const xmlChar *parent_name) +{ + struct others *new = NULL; + struct others *end = head; + + new = calloc(1, sizeof(*new)); + if (new == NULL) { + CU_DEBUG("calloc space failed."); + return NULL; + } + + new->id = 0; + new->name = strdup((char *)name); + new->parent_id = parent_id; + if (parent_name) { + new->parent_name = strdup((char *)parent_name); + } + new->type = type; + if (type == TYPE_PROP) { + new->value = get_attr_value(node, (char *)name); + } else if (type == TYPE_NODE) { + new->value = get_node_content(node); + } + new->next = NULL; + + if (head == NULL) { + head = new; + } else { + /* Seek to the end of link list and calculate conflicts */ + while (end) { + if (STREQ(end->name, (char *)name) && + end->type == type) { + /* id conflict, +1 */ + new->id++; + } + if (end->next) { + end = end->next; + } else { + break; + } + } + end->next = new; + } + new->status = ACTIVE; + + return head; +} + +bool compare_param_int(int a, int b) +{ + if (a == -1 || b == -1) { + return true; + } else if (a == b) { + return true; + } else { + return false; + } +} + +bool compare_param_str(const char *a, const char *b) +{ + if (a == NULL || b == NULL) { + return true; + } else if (STREQ(a, b)) { + return true; + } else { + return false; + } +} + +/* + * This function is the main operation for members of virtual device structure. + * The whole data of xml is saved in the others link list. By calling this + * function each variable of virtual device could get the value it wanted. + * After value was fetched, the node in the others link list to be INACTIVE + * status. + * + * @head: The link list saved all data needed. + * @id: Id of name, to identify different nodes but with same name. + * @name: Name of node the caller want to get value. + * @type: TYPE_NODE: Caller want to get a value of node. + * TYPE_PROP: Caller want to get a value of attribute. + * @parent_id: The id of parent node caller want. + * @parent_name: The name of parent node caller want. + * + * If caller doesn't want to point @id, @parent_id or @parent_name, + * @id could be set as -1, @parent_id and @parent_name should be + * set as NULL. + */ +char *fetch_from_others(struct others **head, + int id, + char *name, + enum others_type type, + int parent_id, + char *parent_name) +{ + struct others *tmp = *head; + char *value = NULL; + + while (tmp) { + if (compare_param_int(tmp->id, id) && + STREQ(tmp->name, name) && + compare_param_int(tmp->parent_id, parent_id) && + compare_param_str(tmp->parent_name, parent_name) && + tmp->type == type && + tmp->status == ACTIVE) { + value = strdup(tmp->value); + tmp->status = INACTIVE; + return value; + } + tmp = tmp->next; + } + + return NULL; +} + +/* + * This function seeks in the others link list and return the result of + * seeking. If success the node seeked will to be INACTIVE status. + * + * @head: The link list head pointer to be seeked. + * @id: Id of node to be seeked. + * @name: Name of node to be seeked. + * @type: TYPE_NODE: The node to be seeked is a node of xml. + * TYPE_PROP: The node to be seeked is a attribute of a node. + * @parent_id: Id of parent node which to be seeked. + * @parent_name: Name of parent node which to be seeked. + * + * If the caller doesn't point @id, @parent_id or @parent_name, they should + * be set as -1, -1 or NULL. Other parameters is mandatory. + */ +static bool seek_in_others(struct others **head, + int id, + char *name, + enum others_type type, + int parent_id, + char *parent_name) +{ + struct others *tmp = *head; + + while (tmp) { + if (compare_param_int(tmp->id, id) && + STREQ(tmp->name, name) && + compare_param_int(tmp->parent_id, parent_id) && + compare_param_str(tmp->parent_name, parent_name) && + tmp->type == type && + tmp->status == ACTIVE) { + tmp->status = INACTIVE; + return true; + } + tmp = tmp->next; + } + + return false; +} + +struct others *combine_others(struct others *head1, + struct others *head2) +{ + struct others *tail1 = head1; + + if (tail1 == NULL) { + return head2; + } + + while (tail1->next) { + tail1 = tail1->next; + } + + tail1->next = head2; + return head1; +} + +static struct others *seek_to_tail(struct others *others) +{ + if (others == NULL) { + return NULL; + } + + while (others->next) { + others = others->next; + } + + return others; +} + +/* + * Parse all data from xml and build a link list to save it. + * + * @head: Where data from xml to save. + * @node: The root node of xml to be parsed. + * @parent_id: Parent node id of root node, usually zero. + * @parent_name: Parent node name of root node. + */ +static struct others *parse_data_to_others(struct others *head, + xmlNode *node, + int parent_id, + const xmlChar *parent_name) +{ + xmlNode *child = NULL; + xmlAttrPtr attrPtr = NULL; + struct others *tail = NULL; + int tail_id = 0; + + /* If name of node is "text", all operations will skip */ + if (XSTREQ(node->name, "text")) { + return head; + } + + head = add_others(head, + node, + node->name, + TYPE_NODE, + parent_id, + parent_name); + + if (head == NULL) { + goto err; + } + + /* tail is the node we added above */ + tail = seek_to_tail(head); + if (tail == NULL) { + tail_id = 0; + } else { + tail_id = tail->id; + } + + /* Get properties of node */ + attrPtr = node->properties; + while (attrPtr) { + head = add_others(head, + node, + attrPtr->name, + TYPE_PROP, + tail_id, + node->name); + if (head == NULL) { + goto err; + } + + attrPtr = attrPtr->next; + } + + for (child = node->children; child != NULL; child = child->next) { + /* Recursion to restore child's properties or child if have */ + head = parse_data_to_others(head, child, tail_id, node->name); + } + + return head; +err: + CU_DEBUG("add_others failed."); + return NULL; +} + static int parse_fs_device(xmlNode *dnode, struct virt_device **vdevs) { struct virt_device *vdev = NULL; diff --git a/libxkutil/device_parsing.h b/libxkutil/device_parsing.h index 603f29a..92d8638 100644 --- a/libxkutil/device_parsing.h +++ b/libxkutil/device_parsing.h @@ -299,6 +299,23 @@ int attach_device(virDomainPtr dom, struct virt_device *dev); int detach_device(virDomainPtr dom, struct virt_device *dev); int change_device(virDomainPtr dom, struct virt_device *dev); +char *fetch_from_others(struct others **head, + int id, + char *name, + enum others_type type, + int parent_id, + char *parent_name); + +bool compare_parent(const char *a, const char *b); + +void print_others(struct others *head); + +struct others *combine_others(struct others *head1, struct others *head2); + +bool compare_param_int(int a, int b); + +bool compare_param_str(const char *a, const char *b); + #define XSTREQ(x, y) (STREQ((char *)x, y)) #define STRPROP(d, p, n) (d->p = get_node_content(n)) -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:23:58 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:23:58 -0500 Subject: [Libvirt-cim] [PATCH 09/20] Fix xml parsing algorithm for parse_emu_device() In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-10-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.c | 24 +++++++++++++++++++++--- libxkutil/device_parsing.h | 1 + 2 files changed, 22 insertions(+), 3 deletions(-) diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c index 8082b37..5fdf28e 100644 --- a/libxkutil/device_parsing.c +++ b/libxkutil/device_parsing.c @@ -165,6 +165,7 @@ static void cleanup_emu_device(struct emu_device *dev) return; free(dev->path); + cleanup_others(dev->others); } static void cleanup_vnc_device(struct graphics_device *dev) @@ -1506,15 +1507,32 @@ static int parse_emu_device(xmlNode *node, struct virt_device **vdevs) struct virt_device *vdev = NULL; struct emu_device *edev = NULL; + CU_DEBUG("Enter parse_emu_device()."); + vdev = calloc(1, sizeof(*vdev)); - if (vdev == NULL) + if (vdev == NULL) { + CU_DEBUG("calloc failed."); goto err; + } edev = &(vdev->dev.emu); - edev->path = get_node_content(node); - if (edev->path == NULL) + edev->others = parse_data_to_others(edev->others, + node, + 0, + BAD_CAST "devices"); + + edev->path = fetch_from_others(&edev->others, + -1, + (char *)node->name, + TYPE_NODE, + -1, + "devices"); + + if (edev->path == NULL) { + CU_DEBUG("no path"); goto err; + } vdev->type = CIM_RES_TYPE_EMU; diff --git a/libxkutil/device_parsing.h b/libxkutil/device_parsing.h index c9e2b29..c5517e3 100644 --- a/libxkutil/device_parsing.h +++ b/libxkutil/device_parsing.h @@ -120,6 +120,7 @@ struct vcpu_device { struct emu_device { char *path; + struct others *others; }; struct vnc_device { -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:23:55 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:23:55 -0500 Subject: [Libvirt-cim] [PATCH 06/20] Fix xml parsing algorithm for parse_vsi_device() In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-7-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.c | 89 ++++++++++++++++++++++++++++++++++------------ 1 file changed, 66 insertions(+), 23 deletions(-) diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c index dac2ea1..940b105 100644 --- a/libxkutil/device_parsing.c +++ b/libxkutil/device_parsing.c @@ -1129,40 +1129,83 @@ static int parse_disk_device(xmlNode *dnode, struct virt_device **vdevs) } } -static int parse_vsi_device(xmlNode *dnode, struct net_device *vdevs) +static int parse_vsi_device(struct others **others, struct net_device *vdevs) { struct vsi_device *vsi_dev = NULL; - xmlNode * child = NULL; + + CU_DEBUG("Enter parse_vsi_device()."); vsi_dev = calloc(1, sizeof(*vsi_dev)); - if (vsi_dev == NULL) + if (vsi_dev == NULL) { + CU_DEBUG("calloc failed."); goto err; + } - vsi_dev->vsi_type = get_attr_value(dnode, "type"); - if (vsi_dev->vsi_type == NULL) + vsi_dev->vsi_type = fetch_from_others(others, + -1, + "type", + TYPE_PROP, + -1, + "virtualport"); + if (vsi_dev->vsi_type == NULL) { + CU_DEBUG("no vsi type"); goto err; + } - for (child = dnode->children; child != NULL; child = child->next) { - if (XSTREQ(child->name, "parameters")) { - vsi_dev->manager_id = get_attr_value(child, - "managerid"); - if (vsi_dev->manager_id == NULL) - goto err; + if (seek_in_others(others, + -1, + "parameters", + TYPE_NODE, + -1, + "virtualport")) { + vsi_dev->manager_id = fetch_from_others(others, + -1, + "managerid", + TYPE_PROP, + -1, + "parameters"); + + if (vsi_dev->manager_id == NULL) { + CU_DEBUG("no managerid"); + goto err; + } - vsi_dev->type_id = get_attr_value(child, "typeid"); - if (vsi_dev->type_id == NULL) - goto err; + vsi_dev->type_id = fetch_from_others(others, + -1, + "typeid", + TYPE_PROP, + -1, + "parameters"); + if (vsi_dev->type_id == NULL) { + CU_DEBUG("no typeid"); + goto err; + } - vsi_dev->type_id_version = - get_attr_value(child, "typeidversion"); - if (vsi_dev->type_id_version == NULL) - goto err; + vsi_dev->type_id_version = fetch_from_others(others, + -1, + "typeidversion", + TYPE_PROP, + -1, + "parameters"); - vsi_dev->instance_id = get_attr_value(child, - "instanceid"); - vsi_dev->profile_id = get_attr_value(child, - "profileid"); - } + if (vsi_dev->type_id_version == NULL) { + CU_DEBUG("no typeidversion"); + goto err; + } + + vsi_dev->instance_id = fetch_from_others(others, + -1, + "instanceid", + TYPE_PROP, + -1, + "parameters"); + + vsi_dev->profile_id = fetch_from_others(others, + -1, + "profileid", + TYPE_PROP, + -1, + "parameters"); } memcpy(&(vdevs->vsi), vsi_dev, sizeof(*vsi_dev)); -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:23:54 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:23:54 -0500 Subject: [Libvirt-cim] [PATCH 05/20] Fix xml parsing algorithm for parse_block_device() In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-6-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.c | 180 +++++++++++++++++++++++++++++++++++++-------- 1 file changed, 149 insertions(+), 31 deletions(-) diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c index 20b35e1..dac2ea1 100644 --- a/libxkutil/device_parsing.c +++ b/libxkutil/device_parsing.c @@ -925,7 +925,8 @@ static int parse_block_device(xmlNode *dnode, struct virt_device **vdevs) { struct virt_device *vdev = NULL; struct disk_device *ddev = NULL; - xmlNode * child = NULL; + + CU_DEBUG("Enter parse_block_device()."); vdev = calloc(1, sizeof(*vdev)); if (vdev == NULL) @@ -933,47 +934,164 @@ static int parse_block_device(xmlNode *dnode, struct virt_device **vdevs) ddev = &(vdev->dev.disk); - ddev->type = get_attr_value(dnode, "type"); - if (ddev->type == NULL) + ddev->others = parse_data_to_others(ddev->others, + dnode, + 0, + BAD_CAST "devices"); + if (ddev->others == NULL) { + CU_DEBUG("xml file parse failed."); goto err; + } + + /* fetch out tag from others. It will be removed + * after others management finished. */ + fetch_from_others(&ddev->others, + -1, + "disk", + TYPE_NODE, + -1, + "devices"); - ddev->device = get_attr_value(dnode, "device"); - if (ddev->device == NULL) + ddev->type = fetch_from_others(&ddev->others, + -1, + "type", + TYPE_PROP, + -1, + (char *)dnode->name); + if (ddev->type == NULL) { + CU_DEBUG("no type"); goto err; + } - for (child = dnode->children; child != NULL; child = child->next) { - if (XSTREQ(child->name, "driver")) { - ddev->driver = get_attr_value(child, "name"); - if (ddev->driver == NULL) + ddev->device = fetch_from_others(&ddev->others, + -1, + "device", + TYPE_PROP, + -1, + (char *)dnode->name); + if (ddev->device == NULL) { + CU_DEBUG("no device"); + goto err; + } + + if (seek_in_others(&ddev->others, + -1, + "driver", + TYPE_NODE, + -1, + (char *)dnode->name)) { + ddev->driver = fetch_from_others(&ddev->others, + -1, + "name", + TYPE_PROP, + -1, + "driver"); + + if (ddev->driver == NULL) { + CU_DEBUG("no driver name"); + goto err; + } + + ddev->driver_type = fetch_from_others(&ddev->others, + -1, + "type", + TYPE_PROP, + -1, + "driver"); + + ddev->cache = fetch_from_others(&ddev->others, + -1, + "cache", + TYPE_PROP, + -1, + "driver"); + } + + if (seek_in_others(&ddev->others, + -1, + "source", + TYPE_NODE, + -1, + (char *)dnode->name)) { + ddev->source = fetch_from_others(&ddev->others, + -1, + "file", + TYPE_PROP, + -1, + "source"); + if (ddev->source == NULL) { + ddev->source = fetch_from_others(&ddev->others, + -1, + "dev", + TYPE_PROP, + -1, + "source"); + + if (ddev->source == NULL) { + CU_DEBUG("no source file/dev"); goto err; - ddev->driver_type = get_attr_value(child, "type"); - ddev->cache = get_attr_value(child, "cache"); - } else if (XSTREQ(child->name, "source")) { - ddev->source = get_attr_value(child, "file"); - if (ddev->source) { - ddev->disk_type = DISK_FILE; - continue; - } - ddev->source = get_attr_value(child, "dev"); - if (ddev->source) { + } else { ddev->disk_type = DISK_PHY; - continue; } + } else { + ddev->disk_type = DISK_FILE; + } + } + + if (seek_in_others(&ddev->others, + -1, + "target", + TYPE_NODE, + -1, + (char *)dnode->name)) { + ddev->virtual_dev = fetch_from_others(&ddev->others, + -1, + "dev", + TYPE_PROP, + -1, + "target"); + + if (ddev->virtual_dev == NULL) { + CU_DEBUG("no target dev"); goto err; - } else if (XSTREQ(child->name, "target")) { - ddev->virtual_dev = get_attr_value(child, "dev"); - if (ddev->virtual_dev == NULL) - goto err; - ddev->bus_type = get_attr_value(child, "bus"); - } else if (XSTREQ(child->name, "readonly")) { - ddev->readonly = true; - } else if (XSTREQ(child->name, "shareable")) { - ddev->shareable = true; - } else if (XSTREQ(child->name, "address")) { - parse_device_address(child, &ddev->address); } + + ddev->bus_type = fetch_from_others(&ddev->others, + -1, + "bus", + TYPE_PROP, + -1, + "target"); } + ddev->readonly = seek_in_others(&ddev->others, + -1, + "readonly", + TYPE_NODE, + -1, + (char *)dnode->name); + + ddev->shareable = seek_in_others(&ddev->others, + -1, + "shareable", + TYPE_NODE, + -1, + (char *)dnode->name); + + if (seek_in_others(&ddev->others, + -1, + "address", + TYPE_NODE, + -1, + (char *)dnode->name)) { + if (!fetch_device_address_from_others(&ddev->others, + &ddev->address)) { + CU_DEBUG("error fetching device address"); + goto err; + } + } + + /* handle the situation that a cdrom device have no disk in it, no ISO file */ if ((XSTREQ(ddev->device, "cdrom")) && (ddev->source == NULL)) { ddev->source = strdup(""); -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:23:59 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:23:59 -0500 Subject: [Libvirt-cim] [PATCH 10/20] Fix xml parsing algorithm for parse_mem_device() In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-11-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.c | 52 +++++++++++++++++++++++++++++++++++++++++----- libxkutil/device_parsing.h | 1 + 2 files changed, 48 insertions(+), 5 deletions(-) diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c index 5fdf28e..0a74ff3 100644 --- a/libxkutil/device_parsing.c +++ b/libxkutil/device_parsing.c @@ -159,6 +159,14 @@ static void cleanup_vcpu_device(struct vcpu_device *dev) cleanup_others(dev->others); } +static void cleanup_mem_device(struct mem_device *dev) +{ + if (dev == NULL) + return; + + cleanup_others(dev->others); +} + static void cleanup_emu_device(struct emu_device *dev) { if (dev == NULL) @@ -350,6 +358,8 @@ void cleanup_virt_device(struct virt_device *dev) cleanup_net_device(&dev->dev.net); else if (dev->type == CIM_RES_TYPE_PROC) cleanup_vcpu_device(&dev->dev.vcpu); + else if (dev->type == CIM_RES_TYPE_MEM) + cleanup_mem_device(&dev->dev.mem); else if (dev->type == CIM_RES_TYPE_EMU) cleanup_emu_device(&dev->dev.emu); else if (dev->type == CIM_RES_TYPE_GRAPHICS) @@ -1553,20 +1563,50 @@ static int parse_mem_device(xmlNode *node, struct virt_device **vdevs) char *content = NULL; char *tmpval = NULL; int ret = 0; + struct others *new_others = NULL; + + CU_DEBUG("Enter parse_mem_device()."); vdev = calloc(1, sizeof(*vdev)); - if (vdev == NULL) + if (vdev == NULL) { + CU_DEBUG("calloc failed."); goto err; + } mdev = &(vdev->dev.mem); - content = get_node_content(node); + new_others = parse_data_to_others(new_others, + node, + 0, + BAD_CAST "domain"); + mdev->others = combine_others(mdev->others, new_others); + + if (XSTREQ(node->name, "currentMemory")) { + content = fetch_from_others(&mdev->others, + -1, + "currentMemory", + TYPE_NODE, + -1, + "domain"); - if (XSTREQ(node->name, "currentMemory")) sscanf(content, "%" PRIu64, &mdev->size); - else if (XSTREQ(node->name, "memory")) { + } else if (XSTREQ(node->name, "memory")) { + content = fetch_from_others(&mdev->others, + -1, + "memory", + TYPE_NODE, + -1, + "domain"); + sscanf(content, "%" PRIu64, &mdev->maxsize); - tmpval = get_attr_value(node, "dumpCore"); + + tmpval = fetch_from_others(&mdev->others, + -1, + "dumpCore", + TYPE_PROP, + -1, + "memory"); + if (tmpval && XSTREQ(tmpval, "on")) { mdev->dumpCore = MEM_DUMP_CORE_ON; } else if (tmpval && XSTREQ(content, "off")) { @@ -1574,6 +1614,8 @@ static int parse_mem_device(xmlNode *node, struct virt_device **vdevs) } else { mdev->dumpCore = MEM_DUMP_CORE_NOT_SET; } + } else { + /* do nothing */ } *vdevs = vdev; diff --git a/libxkutil/device_parsing.h b/libxkutil/device_parsing.h index c5517e3..0855166 100644 --- a/libxkutil/device_parsing.h +++ b/libxkutil/device_parsing.h @@ -109,6 +109,7 @@ struct mem_device { enum { MEM_DUMP_CORE_NOT_SET, MEM_DUMP_CORE_ON, MEM_DUMP_CORE_OFF } dumpCore; + struct others *others; }; struct vcpu_device { -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:23:57 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:23:57 -0500 Subject: [Libvirt-cim] [PATCH 08/20] Fix xml parsing algorithm for parse_vcpu_device() In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-9-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.c | 35 ++++++++++++++++++++++++++++++----- libxkutil/device_parsing.h | 1 + 2 files changed, 31 insertions(+), 5 deletions(-) diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c index 8b39cda..8082b37 100644 --- a/libxkutil/device_parsing.c +++ b/libxkutil/device_parsing.c @@ -151,6 +151,14 @@ static void cleanup_net_device(struct net_device *dev) cleanup_others(dev->others); } +static void cleanup_vcpu_device(struct vcpu_device *dev) +{ + if (dev == NULL) + return; + + cleanup_others(dev->others); +} + static void cleanup_emu_device(struct emu_device *dev) { if (dev == NULL) @@ -339,6 +347,8 @@ void cleanup_virt_device(struct virt_device *dev) cleanup_disk_device(&dev->dev.disk); else if (dev->type == CIM_RES_TYPE_NET) cleanup_net_device(&dev->dev.net); + else if (dev->type == CIM_RES_TYPE_PROC) + cleanup_vcpu_device(&dev->dev.vcpu); else if (dev->type == CIM_RES_TYPE_EMU) cleanup_emu_device(&dev->dev.emu); else if (dev->type == CIM_RES_TYPE_GRAPHICS) @@ -1450,7 +1460,26 @@ static int parse_vcpu_device(xmlNode *node, struct virt_device **vdevs) char *count_str; int count; - count_str = get_node_content(node); + CU_DEBUG("Enter parse_vcpu_device()."); + + list = calloc(1, sizeof(*list)); + if (list == NULL) { + CU_DEBUG("calloc failed."); + goto err; + } + + list->dev.vcpu.others = parse_data_to_others(list->dev.vcpu.others, + node, + 0, + BAD_CAST "domain"); + + count_str = fetch_from_others(&list->dev.vcpu.others, + -1, + "vcpu", + TYPE_NODE, + -1, + "domain"); + if (count_str == NULL) count = 1; /* Default to 1 VCPU if non specified */ else if (sscanf(count_str, "%i", &count) != 1) @@ -1458,10 +1487,6 @@ static int parse_vcpu_device(xmlNode *node, struct virt_device **vdevs) free(count_str); - list = calloc(1, sizeof(*list)); - if (list == NULL) - goto err; - list->dev.vcpu.quantity = count; list->type = CIM_RES_TYPE_PROC; diff --git a/libxkutil/device_parsing.h b/libxkutil/device_parsing.h index 4d26fc8..c9e2b29 100644 --- a/libxkutil/device_parsing.h +++ b/libxkutil/device_parsing.h @@ -115,6 +115,7 @@ struct vcpu_device { uint64_t quantity; uint32_t weight; uint64_t limit; + struct others *others; }; struct emu_device { -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:23:56 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:23:56 -0500 Subject: [Libvirt-cim] [PATCH 07/20] Fix xml parsing algorithm for parse_net_device() In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-8-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.c | 301 +++++++++++++++++++++++++++++++-------------- libxkutil/device_parsing.h | 1 + 2 files changed, 211 insertions(+), 91 deletions(-) diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c index 940b105..8b39cda 100644 --- a/libxkutil/device_parsing.c +++ b/libxkutil/device_parsing.c @@ -148,6 +148,7 @@ static void cleanup_net_device(struct net_device *dev) free(dev->poolid); cleanup_vsi_device(&dev->vsi); cleanup_device_address(&dev->address); + cleanup_others(dev->others); } static void cleanup_emu_device(struct emu_device *dev) @@ -429,30 +430,6 @@ int add_device_address_property(struct device_address *devaddr, return 0; } - -static int parse_device_address(xmlNode *anode, struct device_address *devaddr) -{ - xmlAttr *attr = NULL; - char *name = NULL; - char *value = NULL; - - for (attr = anode->properties; attr != NULL; attr = attr->next) { - name = (char*) attr->name; - value = get_attr_value(anode, name); - if (!add_device_address_property(devaddr, name, value)) - goto err; - free(value); - } - - return 1; - - err: - cleanup_device_address(devaddr); - free(value); - - return 0; -} - /* * Device addresses can have many forms based on the 'type' field, so rather * than try to create a function that could parse each particular option, we @@ -1222,92 +1199,234 @@ static int parse_net_device(xmlNode *inode, struct virt_device **vdevs) { struct virt_device *vdev = NULL; struct net_device *ndev = NULL; - xmlNode *child = NULL; + + CU_DEBUG("Enter parse_net_device()."); vdev = calloc(1, sizeof(*vdev)); - if (vdev == NULL) + if (vdev == NULL) { + CU_DEBUG("calloc failed."); goto err; + } ndev = &(vdev->dev.net); - ndev->type = get_attr_value(inode, "type"); - if (ndev->type == NULL) + ndev->others = parse_data_to_others(ndev->others, + inode, + 0, + BAD_CAST "devices"); + if (ndev->others == NULL) { + CU_DEBUG("parse xml data to others failed."); goto err; + } + + /* fetch out tag from others. It will be removed + * after others management finished. */ + fetch_from_others(&ndev->others, + -1, + "interface", + TYPE_NODE, + -1, + "devices"); + + ndev->type = fetch_from_others(&ndev->others, + -1, + "type", + TYPE_PROP, + -1, + (char *)inode->name); + + if (ndev->type == NULL) { + CU_DEBUG("no type"); + goto err; + } + + if (seek_in_others(&ndev->others, + -1, + "mac", + TYPE_NODE, + -1, + (char *)inode->name)) { + ndev->mac = fetch_from_others(&ndev->others, + -1, + "address", + TYPE_PROP, + -1, + "mac"); + + if (ndev->mac == NULL) { + CU_DEBUG("no mac address"); + goto err; + } + } + + if (seek_in_others(&ndev->others, + -1, + "source", + TYPE_NODE, + -1, + (char *)inode->name)) { + ndev->source = fetch_from_others(&ndev->others, + -1, + "bridge", + TYPE_PROP, + -1, + "source"); + + if (ndev->source == NULL) { + ndev->source = fetch_from_others(&ndev->others, + -1, + "network", + TYPE_PROP, + -1, + "source"); - for (child = inode->children; child != NULL; child = child->next) { - if (XSTREQ(child->name, "mac")) { - ndev->mac = get_attr_value(child, "address"); - if (ndev->mac == NULL) - goto err; - } else if (XSTREQ(child->name, "source")) { - ndev->source = get_attr_value(child, "bridge"); - if (ndev->source != NULL) - continue; - ndev->source = get_attr_value(child, "network"); if (ndev->source != NULL) { - int ret = asprintf(&ndev->poolid, + int ret = asprintf(&ndev->poolid, "NetworkPool/%s", ndev->source); + if (ret == -1) { - CU_DEBUG("Failed to get network" - " poolid"); + CU_DEBUG("Failed to get network poolid"); + goto err; } - continue; + } else { + ndev->source = fetch_from_others(&ndev->others, + -1, + "dev", + TYPE_PROP, + -1, + "source"); + + ndev->net_mode = fetch_from_others(&ndev->others, + -1, + "mode", + TYPE_PROP, + -1, + "source"); + + if ((ndev->source == NULL) || (ndev->net_mode == NULL)) { + CU_DEBUG("source %s, mode %s", ndev->source, ndev->net_mode); + goto err; + } + } - ndev->source = get_attr_value(child, "dev"); - ndev->net_mode = get_attr_value(child, "mode"); - if ((ndev->source != NULL) && (ndev->net_mode != NULL)) - continue; + } + } + + if (seek_in_others(&ndev->others, + -1, + "target", + TYPE_NODE, + -1, + (char *)inode->name)) { + ndev->device = fetch_from_others(&ndev->others, + -1, + "dev", + TYPE_PROP, + -1, + "target"); + + if (ndev->device == NULL) { + CU_DEBUG("no dev in target."); goto err; - } else if (XSTREQ(child->name, "target")) { - ndev->device = get_attr_value(child, "dev"); - if (ndev->device == NULL) - goto err; - } else if (XSTREQ(child->name, "model")) { - ndev->model = get_attr_value(child, "type"); - if (ndev->model == NULL) - goto err; - } else if (XSTREQ(child->name, "filterref")) { - ndev->filter_ref = get_attr_value(child, "filter"); - } else if (XSTREQ(child->name, "virtualport")) { - parse_vsi_device(child, ndev); - } else if (XSTREQ(child->name, "address")) { - parse_device_address(child, &ndev->address); + } + } + + if (seek_in_others(&ndev->others, + -1, + "model", + TYPE_NODE, + -1, + (char *)inode->name)) { + ndev->model = fetch_from_others(&ndev->others, + -1, + "type", + TYPE_PROP, + -1, + "model"); + if (ndev->model == NULL) { + CU_DEBUG("no model type."); + goto err; + } + } + + if (seek_in_others(&ndev->others, + -1, + "filterref", + TYPE_NODE, + -1, + (char *)inode->name)) { + ndev->filter_ref = fetch_from_others(&ndev->others, + -1, + "filter", + TYPE_PROP, + -1, + "filterref"); + } + + if (seek_in_others(&ndev->others, -1, "virtualport", + TYPE_NODE, -1, (char *)inode->name)) { + parse_vsi_device(&ndev->others, ndev); + } + + if (seek_in_others(&ndev->others, + -1, + "address", + TYPE_NODE, + -1, + (char *)inode->name)) { + if (!fetch_device_address_from_others(&ndev->others, + &ndev->address)) { + CU_DEBUG("error fetching device address"); + goto err; + } + } + #if LIBVIR_VERSION_NUMBER >= 9000 - } else if (XSTREQ(child->name, "bandwidth")) { - /* Network QoS bandwidth support */ - xmlNode *grandchild = NULL; - for (grandchild = child->children; - grandchild != NULL; - grandchild = grandchild->next) { - if (XSTREQ(grandchild->name, "inbound")) { - /* Only expose inbound bandwidth */ - char *val; - - val = get_attr_value(grandchild, - "average"); - if (val != NULL) { - sscanf(val, "%" PRIu64, - &ndev->reservation); - free(val); - } else - ndev->reservation = 0; - - val = get_attr_value(grandchild, - "peak"); - if (val != NULL) { - sscanf(val, "%" PRIu64, - &ndev->limit); - free(val); - } else - ndev->limit = 0; - break; - } - } + /* Network QoS bandwidth support */ + /* Only expose inbound bandwidth */ + char *val; + if (seek_in_others(&ndev->others, + -1, + "bandwidth", + TYPE_NODE, + -1, + (char *)inode->name) && + seek_in_others(&ndev->others, + -1, + "inbound", + TYPE_NODE, + -1, + "bandwidth")) { + val = fetch_from_others(&ndev->others, + -1, + "average", + TYPE_PROP, + -1, + "inbound"); + + if (val != NULL) { + sscanf(val, "%" PRIu64, &ndev->reservation); + free(val); + } else { + ndev->reservation = 0; } -#endif + val = fetch_from_others(&ndev->others, + -1, + "peak", + TYPE_PROP, + -1, + "inbound"); + + if (val != NULL) { + sscanf(val, "%" PRIu64, &ndev->limit); + free(val); + } else { + ndev->limit = 0; + } } +#endif if (ndev->source == NULL) CU_DEBUG("No network source defined, leaving blank\n"); diff --git a/libxkutil/device_parsing.h b/libxkutil/device_parsing.h index 6e8fe25..4d26fc8 100644 --- a/libxkutil/device_parsing.h +++ b/libxkutil/device_parsing.h @@ -100,6 +100,7 @@ struct net_device { uint64_t limit; struct vsi_device vsi; struct device_address address; + struct others *others; }; struct mem_device { -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:24:02 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:24:02 -0500 Subject: [Libvirt-cim] [PATCH 13/20] Fix xml parsing algorithm for parse_input_device() In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-14-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.c | 41 ++++++++++++++++++++++++++++++++++++++--- libxkutil/device_parsing.h | 1 + 2 files changed, 39 insertions(+), 3 deletions(-) diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c index c77f3e5..5a012bc 100644 --- a/libxkutil/device_parsing.c +++ b/libxkutil/device_parsing.c @@ -347,6 +347,7 @@ static void cleanup_input_device(struct input_device *dev) free(dev->type); free(dev->bus); + cleanup_others(dev->others); } void cleanup_virt_device(struct virt_device *dev) @@ -2087,14 +2088,48 @@ static int parse_input_device(xmlNode *node, struct virt_device **vdevs) struct input_device *idev = NULL; int ret; + CU_DEBUG("Enter parse_input_device()."); + vdev = calloc(1, sizeof(*vdev)); - if (vdev == NULL) + if (vdev == NULL) { + CU_DEBUG("calloc failed."); goto err; + } idev = &(vdev->dev.input); - idev->type = get_attr_value(node, "type"); - idev->bus = get_attr_value(node, "bus"); + idev->others = parse_data_to_others(idev->others, + node, + 0, + BAD_CAST "devices"); + + if (idev->others == NULL) { + CU_DEBUG("parse data to others failed."); + goto err; + } + + /* fetch out tag from others. It will be removed + * after others management finished. */ + fetch_from_others(&idev->others, + -1, + "input", + TYPE_NODE, + -1, + "devices"); + + idev->type = fetch_from_others(&idev->others, + -1, + "type", + TYPE_PROP, + -1, + (char *)node->name); + + idev->bus = fetch_from_others(&idev->others, + -1, + "bus", + TYPE_PROP, + -1, + (char *)node->name); if ((idev->type == NULL) || (idev->bus == NULL)) goto err; diff --git a/libxkutil/device_parsing.h b/libxkutil/device_parsing.h index b29e58c..275d91f 100644 --- a/libxkutil/device_parsing.h +++ b/libxkutil/device_parsing.h @@ -187,6 +187,7 @@ struct console_device { struct input_device { char *type; char *bus; + struct others *others; }; struct virt_device { -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:24:01 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:24:01 -0500 Subject: [Libvirt-cim] [PATCH 12/20] Fix xml parsing algorithm for parse_graphics_device() In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-13-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.c | 155 +++++++++++++++++++++++++++++++++++---------- libxkutil/device_parsing.h | 1 + 2 files changed, 121 insertions(+), 35 deletions(-) diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c index 8edd2e3..c77f3e5 100644 --- a/libxkutil/device_parsing.c +++ b/libxkutil/device_parsing.c @@ -202,6 +202,7 @@ static void cleanup_graphics_device(struct graphics_device *dev) cleanup_vnc_device(dev); free(dev->type); + cleanup_others(dev->others); } static void cleanup_path_device(struct path_device *dev) @@ -1631,17 +1632,6 @@ static int parse_mem_device(xmlNode *node, struct virt_device **vdevs) return ret; } -static char *get_attr_value_default(xmlNode *node, char *attrname, - const char *default_value) -{ - char *ret = get_attr_value(node, attrname); - - if (ret == NULL && default_value != NULL) - ret = strdup(default_value); - - return ret; -} - static int parse_console_device(xmlNode *node, struct virt_device **vdevs) { struct virt_device *vdev = NULL; @@ -1916,28 +1906,91 @@ static int parse_graphics_device(xmlNode *node, struct virt_device **vdevs) { struct virt_device *vdev = NULL; struct graphics_device *gdev = NULL; - xmlNode *child = NULL; int ret; + CU_DEBUG("Enter parse_graphics_device()."); + vdev = calloc(1, sizeof(*vdev)); - if (vdev == NULL) + if (vdev == NULL) { + CU_DEBUG("calloc failed."); goto err; + } gdev = &(vdev->dev.graphics); - gdev->type = get_attr_value(node, "type"); - if (gdev->type == NULL) + gdev->others = parse_data_to_others(gdev->others, + node, + 0, + BAD_CAST "devices"); + if (gdev->others == NULL) { + CU_DEBUG("parse data to others failed."); goto err; + } + + /* fetch out tag from others. It will be removed + * after others management finished. */ + fetch_from_others(&gdev->others, + -1, + "graphics", + TYPE_NODE, + -1, + "devices"); + fetch_from_others(&gdev->others, + -1, + "serial", + TYPE_NODE, + -1, + "devices"); + + gdev->type = fetch_from_others(&gdev->others, + -1, + "type", + TYPE_PROP, + -1, + (char *)node->name); + if (gdev->type == NULL) { + CU_DEBUG("no type"); + goto err; + } CU_DEBUG("graphics device type = %s", gdev->type); if (STREQC(gdev->type, "vnc")) { - gdev->dev.vnc.port = get_attr_value_default(node, "port", - "-1"); - gdev->dev.vnc.host = get_attr_value_default(node, "listen", - "127.0.0.1"); - gdev->dev.vnc.keymap = get_attr_value(node, "keymap"); - gdev->dev.vnc.passwd = get_attr_value(node, "passwd"); + gdev->dev.vnc.port = fetch_from_others(&gdev->others, + -1, + "port", + TYPE_PROP, + -1, + (char *)node->name); + + if (gdev->dev.vnc.port == NULL) { + gdev->dev.vnc.port = strdup("-1"); + } + + gdev->dev.vnc.host = fetch_from_others(&gdev->others, + -1, + "listen", + TYPE_PROP, + -1, + (char *)node->name); + + if (gdev->dev.vnc.host == NULL) { + gdev->dev.vnc.host = strdup("127.0.0.1"); + } + + gdev->dev.vnc.keymap = fetch_from_others(&gdev->others, + -1, + "keymap", + TYPE_PROP, + -1, + (char *)node->name); + + gdev->dev.vnc.passwd = fetch_from_others(&gdev->others, + -1, + "passwd", + TYPE_PROP, + -1, + (char *)node->name); if (gdev->dev.vnc.port == NULL || gdev->dev.vnc.host == NULL) { CU_DEBUG("Error vnc port '%p' host '%p'", @@ -1946,27 +1999,59 @@ static int parse_graphics_device(xmlNode *node, struct virt_device **vdevs) } } else if (STREQC(gdev->type, "sdl")) { - gdev->dev.sdl.display = get_attr_value(node, "display"); - gdev->dev.sdl.xauth = get_attr_value(node, "xauth"); - gdev->dev.sdl.fullscreen = get_attr_value(node, "fullscreen"); + gdev->dev.sdl.display = fetch_from_others(&gdev->others, + -1, + "display", + TYPE_PROP, + -1, + (char *)node->name); + + gdev->dev.sdl.xauth = fetch_from_others(&gdev->others, + -1, + "xauth", + TYPE_PROP, + -1, + (char *)node->name); + + gdev->dev.sdl.fullscreen = fetch_from_others(&gdev->others, + -1, + "fullscreen", + TYPE_PROP, + -1, + (char *)node->name); } else if (STREQC(gdev->type, "pty")) { - if (node->name == NULL) - goto err; - /* Change type to serial, console, etc. It will be converted * back in xmlgen.c */ free(gdev->type); gdev->type = strdup((char *)node->name); - for (child = node->children; child != NULL; - child = child->next) { - if (XSTREQ(child->name, "source")) - gdev->dev.vnc.host = get_attr_value(child, "path"); - else if (XSTREQ(child->name, "target")) { - gdev->dev.vnc.port = - get_attr_value(child, "port"); - } + if (seek_in_others(&gdev->others, + -1, + "source", + TYPE_NODE, + -1, + (char *)node->name)) { + gdev->dev.vnc.host = fetch_from_others(&gdev->others, + -1, + "path", + TYPE_PROP, + -1, + "source"); + } + + if (seek_in_others(&gdev->others, + -1, + "target", + TYPE_NODE, + -1, + (char *)node->name)) { + gdev->dev.vnc.port = fetch_from_others(&gdev->others, + -1, + "port", + TYPE_PROP, + -1, + "target"); } } else { diff --git a/libxkutil/device_parsing.h b/libxkutil/device_parsing.h index 24b7c1d..b29e58c 100644 --- a/libxkutil/device_parsing.h +++ b/libxkutil/device_parsing.h @@ -143,6 +143,7 @@ struct graphics_device { struct vnc_device vnc; struct sdl_device sdl; } dev; + struct others *others; }; struct path_device { -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:24:03 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:24:03 -0500 Subject: [Libvirt-cim] [PATCH 14/20] Add parse_unknown_device() In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-15-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.c | 75 ++++++++++++++++++++++++++++++++++++++++++++++ libxkutil/device_parsing.h | 7 +++++ src/svpc_types.h | 1 + 3 files changed, 83 insertions(+) diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c index 5a012bc..a6a6cfe 100644 --- a/libxkutil/device_parsing.c +++ b/libxkutil/device_parsing.c @@ -101,6 +101,15 @@ static void cleanup_others(struct others *others) } } +static void cleanup_unknown_device(struct unknown_device *dev) +{ + if (dev == NULL) + return; + + free(dev->name); + cleanup_others(dev->others); +} + static void cleanup_disk_device(struct disk_device *dev) { if (dev == NULL) @@ -371,6 +380,8 @@ void cleanup_virt_device(struct virt_device *dev) cleanup_input_device(&dev->dev.input); else if (dev->type == CIM_RES_TYPE_CONSOLE) cleanup_console_device(&dev->dev.console); + else if (dev->type == CIM_RES_TYPE_UNKDEV) + cleanup_unknown_device(&dev->dev.unknown); free(dev->id); @@ -2152,6 +2163,70 @@ static int parse_input_device(xmlNode *node, struct virt_device **vdevs) return 0; } +static int parse_unknown_device(xmlNode *node, struct virt_device **vdevs) +{ + struct virt_device *vdev = NULL; + struct unknown_device *udev = NULL; + xmlNode *child = NULL; + + CU_DEBUG("Enter parse_unknown_device()."); + + vdev = calloc(1, sizeof(*vdev)); + if (vdev == NULL) { + CU_DEBUG("calloc failed."); + goto err; + } + + udev = &(vdev->dev.unknown); + + for (child = node->children; child != NULL; child = child->next) { + /* Skip all items parsed in other parse_*() functions. + * Everything here is just to be compatible with old versions. + * Here may need some improvement in the future. + */ + if (XSTREQ(child->name, "disk") || + XSTREQ(child->name, "filesystem") || + XSTREQ(child->name, "interface") || + XSTREQ(child->name, "emulator") || + XSTREQ(child->name, "graphics") || + XSTREQ(child->name, "console") || + XSTREQ(child->name, "serial") || + XSTREQ(child->name, "input")) { + /* Just skip them and do nothing */ + } else { + udev->others = parse_data_to_others(udev->others, + child, + 0, + BAD_CAST "devices"); + } + } + + if (udev->others == NULL) { + CU_DEBUG("no others."); + /* Should this really be an error? or success that + * we didn't find something unknown?!! + */ + goto err; + } + + vdev->type = CIM_RES_TYPE_UNKDEV; + + udev->name = strdup("unknown"); + if (udev->name == NULL) { + CU_DEBUG("Failed to strdup unknown name"); + goto err; + } + + *vdevs = vdev; + + return 1; +err: + cleanup_unknown_device(udev); + free(vdev); + + return 0; +} + static bool resize_devlist(struct virt_device **list, int newsize) { struct virt_device *_list; diff --git a/libxkutil/device_parsing.h b/libxkutil/device_parsing.h index 275d91f..a4e60ee 100644 --- a/libxkutil/device_parsing.h +++ b/libxkutil/device_parsing.h @@ -60,6 +60,12 @@ struct others { } status; }; +/* The structure for saving unknown device */ +struct unknown_device { + char *name; + struct others *others; +}; + struct vsi_device { char *vsi_type; char *manager_id; @@ -201,6 +207,7 @@ struct virt_device { struct graphics_device graphics; struct console_device console; struct input_device input; + struct unknown_device unknown; } dev; char *id; }; diff --git a/src/svpc_types.h b/src/svpc_types.h index 404e428..4928742 100644 --- a/src/svpc_types.h +++ b/src/svpc_types.h @@ -33,6 +33,7 @@ #define CIM_RES_TYPE_GRAPHICS 24 #define CIM_RES_TYPE_INPUT 13 #define CIM_RES_TYPE_UNKNOWN 1000 +#define CIM_RES_TYPE_UNKDEV 1001 #define CIM_RES_TYPE_IMAGE 32768 #define CIM_RES_TYPE_CONSOLE 32769 #define CIM_RES_TYPE_EMU 32770 -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:24:00 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:24:00 -0500 Subject: [Libvirt-cim] [PATCH 11/20] Fix xml parsing algorithm for parse_console_device() In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-12-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.c | 273 ++++++++++++++++++++++++++++++++++----------- libxkutil/device_parsing.h | 1 + 2 files changed, 207 insertions(+), 67 deletions(-) diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c index 0a74ff3..8edd2e3 100644 --- a/libxkutil/device_parsing.c +++ b/libxkutil/device_parsing.c @@ -287,6 +287,7 @@ static void cleanup_console_device(struct console_device *dev) dev->source_type = 0; free(dev->target_type); memset(&dev->source_dev, 0, sizeof(dev->source_dev)); + cleanup_others(dev->others); }; static void console_device_dup(struct console_device *t, @@ -1648,18 +1649,37 @@ static int parse_console_device(xmlNode *node, struct virt_device **vdevs) char *source_type_str = NULL; char *target_port_ID = NULL; char *udp_source_mode = NULL; + int id; - xmlNode *child = NULL; + CU_DEBUG("Enter parse_console_device()."); vdev = calloc(1, sizeof(*vdev)); - if (vdev == NULL) + if (vdev == NULL) { + CU_DEBUG("calloc failed."); goto err; + } cdev = &(vdev->dev.console); - source_type_str = get_attr_value(node, "type"); - if (source_type_str == NULL) + cdev->others = parse_data_to_others(cdev->others, + node, + 0, + BAD_CAST "devices"); + if (cdev->others == NULL) { + CU_DEBUG("parse data to others failed."); + goto err; + } + + source_type_str = fetch_from_others(&cdev->others, + -1, + "type", + TYPE_PROP, + -1, + (char *)node->name); + if (source_type_str == NULL) { + CU_DEBUG("no type"); goto err; + } CU_DEBUG("console device type = %s", source_type_str ? : "NULL"); cdev->source_type = chardev_source_type_StrToID(source_type_str); @@ -1668,85 +1688,204 @@ static int parse_console_device(xmlNode *node, struct virt_device **vdevs) CU_DEBUG("console device type ID = %d", cdev->source_type); - for (child = node->children; child != NULL; child = child->next) { - if (XSTREQ(child->name, "target")) { - cdev->target_type = get_attr_value(child, "type"); - CU_DEBUG("Console device target type = '%s'", - cdev->target_type ? : "NULL"); - target_port_ID = get_attr_value(child, "port"); - if (target_port_ID == NULL) - goto err; - } + if (seek_in_others(&cdev->others, + -1, + "target", + TYPE_NODE, + -1, + (char *)node->name)) { + cdev->target_type = fetch_from_others(&cdev->others, + -1, + "type", + TYPE_PROP, + -1, + "target"); + CU_DEBUG("Console device target type = '%s'", + cdev->target_type ? : "NULL"); - if (XSTREQ(child->name, "source")) { - switch (cdev->source_type) - { - case CIM_CHARDEV_SOURCE_TYPE_PTY: - cdev->source_dev.pty.path = - get_attr_value(child, "path"); - break; - case CIM_CHARDEV_SOURCE_TYPE_DEV: - cdev->source_dev.dev.path = - get_attr_value(child, "path"); - break; - case CIM_CHARDEV_SOURCE_TYPE_FILE: - cdev->source_dev.file.path = - get_attr_value(child, "path"); - break; - case CIM_CHARDEV_SOURCE_TYPE_PIPE: - cdev->source_dev.pipe.path = - get_attr_value(child, "path"); - break; - case CIM_CHARDEV_SOURCE_TYPE_UNIXSOCK: - cdev->source_dev.unixsock.mode = - get_attr_value(child, "mode"); - cdev->source_dev.unixsock.path = - get_attr_value(child, "path"); - break; - case CIM_CHARDEV_SOURCE_TYPE_UDP: - udp_source_mode = get_attr_value(child, "mode"); + target_port_ID = fetch_from_others(&cdev->others, + -1, + "port", + TYPE_PROP, + -1, + "target"); + if (target_port_ID == NULL) + goto err; + } + + if (seek_in_others(&cdev->others, + 0, + "source", + TYPE_NODE, + -1, + (char *)node->name)) { + switch (cdev->source_type) + { + case CIM_CHARDEV_SOURCE_TYPE_PTY: + cdev->source_dev.pty.path = + fetch_from_others(&cdev->others, + -1, + "path", + TYPE_PROP, + -1, + "source"); + break; + case CIM_CHARDEV_SOURCE_TYPE_DEV: + cdev->source_dev.dev.path = + fetch_from_others(&cdev->others, + -1, + "path", + TYPE_PROP, + -1, + "source"); + break; + case CIM_CHARDEV_SOURCE_TYPE_FILE: + cdev->source_dev.file.path = + fetch_from_others(&cdev->others, + -1, + "path", + TYPE_PROP, + -1, + "source"); + break; + case CIM_CHARDEV_SOURCE_TYPE_PIPE: + cdev->source_dev.pipe.path = + fetch_from_others(&cdev->others, + -1, + "path", + TYPE_PROP, + -1, + "source"); + break; + case CIM_CHARDEV_SOURCE_TYPE_UNIXSOCK: + cdev->source_dev.unixsock.mode = + fetch_from_others(&cdev->others, + -1, + "mode", + TYPE_PROP, + -1, + "source"); + cdev->source_dev.unixsock.path = + fetch_from_others(&cdev->others, + -1, + "path", + TYPE_PROP, + -1, + "source"); + break; + case CIM_CHARDEV_SOURCE_TYPE_UDP: + /* Something different from other cases. It has two + * tags with mode="bind" and "connect" value. + * Hence here id MUST NOT be ignored. + */ + + /* Just fetch out another tag but we need to do + * nothing with it. + */ + fetch_from_others(&cdev->others, + 1, + "source", + TYPE_NODE, + -1, + "console"); + + for (id = 0; id < 2; id++) { + udp_source_mode = + fetch_from_others(&cdev->others, + -1, + "mode", + TYPE_PROP, + id, + "source"); if (udp_source_mode == NULL) goto err; + if (STREQC(udp_source_mode, "bind")) { cdev->source_dev.udp.bind_host = - get_attr_value(child, "host"); + fetch_from_others(&cdev->others, + -1, + "host", + TYPE_PROP, + id, + "source"); cdev->source_dev.udp.bind_service = - get_attr_value(child, "service"); + fetch_from_others(&cdev->others, + -1, + "service", + TYPE_PROP, + id, + "source"); } else if (STREQC(udp_source_mode, "connect")) { cdev->source_dev.udp.connect_host = - get_attr_value(child, "host"); + fetch_from_others(&cdev->others, + -1, + "host", + TYPE_PROP, + id, + "source"); cdev->source_dev.udp.connect_service = - get_attr_value(child, "service"); + fetch_from_others(&cdev->others, + -1, + "service", + TYPE_PROP, + id, + "source"); } else { CU_DEBUG("unknown udp mode: %s", udp_source_mode ? : "NULL"); goto err; } - break; - case CIM_CHARDEV_SOURCE_TYPE_TCP: - cdev->source_dev.tcp.mode = - get_attr_value(child, "mode"); - cdev->source_dev.tcp.host = - get_attr_value(child, "host"); - cdev->source_dev.tcp.service = - get_attr_value(child, "service"); - break; - - default: - /* Nothing to do for : - CIM_CHARDEV_SOURCE_TYPE_STDIO - CIM_CHARDEV_SOURCE_TYPE_NULL - CIM_CHARDEV_SOURCE_TYPE_VC - CIM_CHARDEV_SOURCE_TYPE_SPICEVMC - */ - break; } + break; + case CIM_CHARDEV_SOURCE_TYPE_TCP: + cdev->source_dev.tcp.mode = + fetch_from_others(&cdev->others, + -1, + "mode", + TYPE_PROP, + -1, + "source"); + cdev->source_dev.tcp.host = + fetch_from_others(&cdev->others, + -1, + "host", + TYPE_PROP, + -1, + "source"); + cdev->source_dev.tcp.service = + fetch_from_others(&cdev->others, + -1, + "service", + TYPE_PROP, + -1, + "source"); + break; + + default: + /* Nothing to do for : + CIM_CHARDEV_SOURCE_TYPE_STDIO + CIM_CHARDEV_SOURCE_TYPE_NULL + CIM_CHARDEV_SOURCE_TYPE_VC + CIM_CHARDEV_SOURCE_TYPE_SPICEVMC + */ + break; } - if ((cdev->source_type == CIM_CHARDEV_SOURCE_TYPE_TCP) - && XSTREQ(child->name, "protocol")) { - cdev->source_dev.tcp.protocol = - get_attr_value(child, "type"); - } + } + + if ((cdev->source_type == CIM_CHARDEV_SOURCE_TYPE_TCP) + && seek_in_others(&cdev->others, + -1, + "protocol", + TYPE_NODE, + -1, + (char *)node->name)) { + cdev->source_dev.tcp.protocol = + fetch_from_others(&cdev->others, + -1, + "type", + TYPE_PROP, + -1, + "protocol"); } vdev->type = CIM_RES_TYPE_CONSOLE; diff --git a/libxkutil/device_parsing.h b/libxkutil/device_parsing.h index 0855166..24b7c1d 100644 --- a/libxkutil/device_parsing.h +++ b/libxkutil/device_parsing.h @@ -180,6 +180,7 @@ struct console_device { struct udp_device udp; } source_dev; char *target_type; + struct others *others; }; struct input_device { -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:24:07 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:24:07 -0500 Subject: [Libvirt-cim] [PATCH 18/20] Fix xml parsing algorithm in parse_features() In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-19-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.c | 38 ++++++++++++++++++++++++++++---------- 1 file changed, 28 insertions(+), 10 deletions(-) diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c index 56c9d3d..43174c1 100644 --- a/libxkutil/device_parsing.c +++ b/libxkutil/device_parsing.c @@ -2774,17 +2774,35 @@ static int parse_os(struct domain *dominfo) return 1; } -static int parse_features(struct domain *dominfo, xmlNode *features) +static int parse_features(struct domain *dominfo) { - xmlNode *child; + CU_DEBUG("Enter parse_features()"); - for (child = features->children; child != NULL; child = child->next) { - if (XSTREQ(child->name, "acpi")) - dominfo->acpi = true; - else if (XSTREQ(child->name, "apic")) - dominfo->apic = true; - else if (XSTREQ(child->name, "pae")) - dominfo->pae = true; + if (seek_in_others(&dominfo->others, + -1, + "acpi", + TYPE_NODE, + -1, + "features")) { + dominfo->acpi = true; + } + + if (seek_in_others(&dominfo->others, + -1, + "apic", + TYPE_NODE, + -1, + "features")) { + dominfo->apic = true; + } + + if (seek_in_others(&dominfo->others, + -1, + "pae", + TYPE_NODE, + -1, + "features")) { + dominfo->pae = true; } return 1; @@ -2943,7 +2961,7 @@ static int parse_domain(xmlNodeSet *nsv, struct domain *dominfo) TYPE_NODE, -1, (char *)nodes[0]->name)) { - /* parse_features(); */ + parse_features(dominfo); } return 1; -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:24:04 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:24:04 -0500 Subject: [Libvirt-cim] [PATCH 15/20] Add parse_devices() for unknown type in get_dominfo_from_xml() In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-16-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.c | 10 ++++++++++ libxkutil/device_parsing.h | 3 +++ 2 files changed, 13 insertions(+) diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c index a6a6cfe..99f6eda 100644 --- a/libxkutil/device_parsing.c +++ b/libxkutil/device_parsing.c @@ -49,6 +49,7 @@ #define GRAPHICS_XPATH (xmlChar *)"/domain/devices/graphics | "\ "/domain/devices/console" #define INPUT_XPATH (xmlChar *)"/domain/devices/input" +#define UNKNOWN_XPATH (xmlChar *)"/domain/devices" #define DEFAULT_BRIDGE "xenbr0" #define DEFAULT_NETWORK "default" @@ -2350,6 +2351,11 @@ static int parse_devices(const char *xml, struct virt_device **_list, int type) func = &parse_input_device; break; + case CIM_RES_TYPE_UNKNOWN: + xpathstr = UNKNOWN_XPATH; + func = &parse_unknown_device; + break; + default: CU_DEBUG("Unrecognized device type. Returning."); goto err1; @@ -2846,6 +2852,9 @@ int get_dominfo_from_xml(const char *xml, struct domain **dominfo) (*dominfo)->dev_vcpu_ct = parse_devices(xml, &(*dominfo)->dev_vcpu, CIM_RES_TYPE_PROC); + (*dominfo)->dev_unknown_ct = parse_devices(xml, + &(*dominfo)->dev_unknown, + CIM_RES_TYPE_UNKNOWN); return ret; @@ -2934,6 +2943,7 @@ void cleanup_dominfo(struct domain **dominfo) cleanup_virt_devices(&dom->dev_graphics, dom->dev_graphics_ct); cleanup_virt_devices(&dom->dev_input, dom->dev_input_ct); cleanup_virt_devices(&dom->dev_console, dom->dev_console_ct); + cleanup_virt_devices(&dom->dev_unknown, dom->dev_unknown_ct); free(dom); diff --git a/libxkutil/device_parsing.h b/libxkutil/device_parsing.h index a4e60ee..fa8aa17 100644 --- a/libxkutil/device_parsing.h +++ b/libxkutil/device_parsing.h @@ -283,6 +283,9 @@ struct domain { struct virt_device *dev_vcpu; int dev_vcpu_ct; + + struct virt_device *dev_unknown; + int dev_unknown_ct; }; struct virt_device *virt_device_dup(struct virt_device *dev); -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:24:05 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:24:05 -0500 Subject: [Libvirt-cim] [PATCH 16/20] Fix xml parsing algorithm in parse_domain() In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-17-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.c | 164 +++++++++++++++++++++++++++++++++++++-------- libxkutil/device_parsing.h | 2 + 2 files changed, 139 insertions(+), 27 deletions(-) diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c index 99f6eda..b244d8a 100644 --- a/libxkutil/device_parsing.c +++ b/libxkutil/device_parsing.c @@ -2736,10 +2736,8 @@ static int parse_features(struct domain *dominfo, xmlNode *features) return 1; } -static void set_action(uint16_t *val, xmlNode *child) +static void set_action(uint16_t *val, const char *action) { - char *action = (char *)xmlNodeGetContent(child); - if (action == NULL) *val = CIM_VSSD_RECOVERY_NONE; else if (STREQ(action, "destroy")) @@ -2750,38 +2748,148 @@ static void set_action(uint16_t *val, xmlNode *child) *val = CIM_VSSD_RECOVERY_RESTART; else *val = CIM_VSSD_RECOVERY_NONE; - - xmlFree(action); } static int parse_domain(xmlNodeSet *nsv, struct domain *dominfo) { xmlNode **nodes = nsv->nodeTab; xmlNode *child; + xmlAttrPtr xmlAttr = NULL; + char *action = NULL; + + CU_DEBUG("Enter parse_domain()"); + + /* parsing attributions of domain into others */ + xmlAttr = nodes[0]->properties; + while(xmlAttr) { + dominfo->others = add_others(dominfo->others, + nodes[0], + xmlAttr->name, + TYPE_PROP, + 0, + nodes[0]->name); + xmlAttr = xmlAttr->next; + } + + dominfo->typestr = fetch_from_others(&dominfo->others, + -1, + "type", + TYPE_PROP, + -1, + (char *)nodes[0]->name); + + /* parse every item in the field of domain and skip some will be parsed + * in other functions. The white list contains: + * (parsed in parse_mem_device()) + * (parsed in parse_mem_device()) + * (parsed in parse_vcpu_device()) + * (parsed in other parse_*_device()) + */ + for (child = nodes[0]->children; child != NULL; child = child->next) { + if (XSTREQ(child->name, "memory") || + XSTREQ(child->name, "currentMemory") || + XSTREQ(child->name, "vcpu") || + XSTREQ(child->name, "devices")) { + continue; + } else { + dominfo->others = parse_data_to_others(dominfo->others, + child, + 0, + nodes[0]->name); + } + } - dominfo->typestr = get_attr_value(nodes[0], "type"); + dominfo->name = fetch_from_others(&dominfo->others, + -1, + "name", + TYPE_NODE, + -1, + (char *)nodes[0]->name); - for (child = nodes[0]->children; child != NULL; child = child->next) { - if (XSTREQ(child->name, "name")) - STRPROP(dominfo, name, child); - else if (XSTREQ(child->name, "uuid")) - STRPROP(dominfo, uuid, child); - else if (XSTREQ(child->name, "bootloader")) - STRPROP(dominfo, bootloader, child); - else if (XSTREQ(child->name, "bootloader_args")) - STRPROP(dominfo, bootloader_args, child); - else if (XSTREQ(child->name, "os")) - parse_os(dominfo, child); - else if (XSTREQ(child->name, "on_poweroff")) - set_action(&dominfo->on_poweroff, child); - else if (XSTREQ(child->name, "on_reboot")) - set_action(&dominfo->on_reboot, child); - else if (XSTREQ(child->name, "on_crash")) - set_action(&dominfo->on_crash, child); - else if (XSTREQ(child->name, "clock")) - dominfo->clock = get_attr_value(child, "offset"); - else if (XSTREQ(child->name, "features")) - parse_features(dominfo, child); + dominfo->uuid = fetch_from_others(&dominfo->others, + -1, + "uuid", + TYPE_NODE, + -1, + (char *)nodes[0]->name); + + dominfo->bootloader = fetch_from_others(&dominfo->others, + -1, + "bootloader", + TYPE_NODE, + -1, + (char *)nodes[0]->name); + + dominfo->bootloader_args = fetch_from_others(&dominfo->others, + -1, + "bootloader_args", + TYPE_NODE, + -1, + (char *)nodes[0]->name); + + if (seek_in_others(&dominfo->others, + -1, + "os", + TYPE_NODE, + -1, + (char *)nodes[0]->name)) { + /* parse_os(); */ + } + + action = fetch_from_others(&dominfo->others, + -1, + "on_poweroff", + TYPE_NODE, + -1, + (char *)nodes[0]->name); + if (action) { + set_action(&dominfo->on_poweroff, action); + free(action); + } + + action = fetch_from_others(&dominfo->others, + -1, + "on_reboot", + TYPE_NODE, + -1, + (char *)nodes[0]->name); + if (action) { + set_action(&dominfo->on_reboot, action); + free(action); + } + + action = fetch_from_others(&dominfo->others, + -1, + "on_crash", + TYPE_NODE, + -1, + (char *)nodes[0]->name); + if (action) { + set_action(&dominfo->on_crash, action); + free(action); + } + + if (seek_in_others(&dominfo->others, + -1, + "clock", + TYPE_NODE, + -1, + (char *)nodes[0]->name)) { + dominfo->clock = fetch_from_others(&dominfo->others, + -1, + "offset", + TYPE_PROP, + -1, + "clock"); + } + + if (seek_in_others(&dominfo->others, + -1, + "features", + TYPE_NODE, + -1, + (char *)nodes[0]->name)) { + /* parse_features(); */ } return 1; @@ -2945,6 +3053,8 @@ void cleanup_dominfo(struct domain **dominfo) cleanup_virt_devices(&dom->dev_console, dom->dev_console_ct); cleanup_virt_devices(&dom->dev_unknown, dom->dev_unknown_ct); + cleanup_others(dom->others); + free(dom); *dominfo = NULL; diff --git a/libxkutil/device_parsing.h b/libxkutil/device_parsing.h index fa8aa17..35011ae 100644 --- a/libxkutil/device_parsing.h +++ b/libxkutil/device_parsing.h @@ -286,6 +286,8 @@ struct domain { struct virt_device *dev_unknown; int dev_unknown_ct; + + struct others *others; }; struct virt_device *virt_device_dup(struct virt_device *dev); -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:24:08 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:24:08 -0500 Subject: [Libvirt-cim] [PATCH 19/20] Add dup function for device copy In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-20-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c index 43174c1..a0ef407 100644 --- a/libxkutil/device_parsing.c +++ b/libxkutil/device_parsing.c @@ -2412,6 +2412,49 @@ static void duplicate_device_address(struct device_address *to, const struct dev cleanup_device_address(to); } +static struct others *dup_others(struct others *others) +{ + struct others *new_node = NULL; + struct others *head = NULL; + + if (others == NULL) { + return NULL; + } + + while (others) { + new_node = calloc(1, sizeof(*new_node)); + if (new_node == NULL) { + CU_DEBUG("calloc failed."); + goto err; + } + + new_node->name = NULL; + new_node->value = NULL; + new_node->parent_name = NULL; + new_node->next = NULL; + + if (head == NULL) { + head = new_node; + } else { + new_node->next = head; + head = new_node; + } + + new_node->id = others->id; + DUP_FIELD(new_node, others, name); + DUP_FIELD(new_node, others, value); + new_node->parent_id = others->parent_id; + DUP_FIELD(new_node, others, parent_name); + new_node->type = others->type; + others = others->next; + } + + return head; +err: + cleanup_others(head); + return NULL; +} + struct virt_device *virt_device_dup(struct virt_device *_dev) { struct virt_device *dev; @@ -2442,6 +2485,7 @@ struct virt_device *virt_device_dup(struct virt_device *_dev) dev->dev.net.reservation = _dev->dev.net.reservation; dev->dev.net.limit = _dev->dev.net.limit; duplicate_device_address(&dev->dev.net.address, &_dev->dev.net.address); + dev->dev.net.others = dup_others(_dev->dev.net.others); } else if (dev->type == CIM_RES_TYPE_DISK) { DUP_FIELD(dev, _dev, dev.disk.type); DUP_FIELD(dev, _dev, dev.disk.device); @@ -2456,25 +2500,32 @@ struct virt_device *virt_device_dup(struct virt_device *_dev) dev->dev.disk.readonly = _dev->dev.disk.readonly; dev->dev.disk.shareable = _dev->dev.disk.shareable; duplicate_device_address(&dev->dev.disk.address, &_dev->dev.disk.address); + dev->dev.disk.others = dup_others(_dev->dev.disk.others); } else if (dev->type == CIM_RES_TYPE_MEM) { dev->dev.mem.size = _dev->dev.mem.size; dev->dev.mem.maxsize = _dev->dev.mem.maxsize; + dev->dev.mem.others = dup_others(_dev->dev.mem.others); } else if (dev->type == CIM_RES_TYPE_PROC) { dev->dev.vcpu.quantity = _dev->dev.vcpu.quantity; + dev->dev.vcpu.others = dup_others(_dev->dev.vcpu.others); } else if (dev->type == CIM_RES_TYPE_EMU) { DUP_FIELD(dev, _dev, dev.emu.path); + dev->dev.emu.others = dup_others(_dev->dev.emu.others); } else if (dev->type == CIM_RES_TYPE_GRAPHICS) { DUP_FIELD(dev, _dev, dev.graphics.type); DUP_FIELD(dev, _dev, dev.graphics.dev.vnc.host); DUP_FIELD(dev, _dev, dev.graphics.dev.vnc.port); DUP_FIELD(dev, _dev, dev.graphics.dev.vnc.keymap); DUP_FIELD(dev, _dev, dev.graphics.dev.vnc.passwd); + dev->dev.graphics.others = dup_others(_dev->dev.graphics.others); } else if (dev->type == CIM_RES_TYPE_INPUT) { DUP_FIELD(dev, _dev, dev.input.type); DUP_FIELD(dev, _dev, dev.input.bus); + dev->dev.input.others = dup_others(_dev->dev.input.others); } else if (dev->type == CIM_RES_TYPE_CONSOLE) { console_device_dup(&dev->dev.console, &_dev->dev.console); + dev->dev.console.others = dup_others(_dev->dev.console.others); } return dev; } -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:24:06 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:24:06 -0500 Subject: [Libvirt-cim] [PATCH 17/20] Fix xml parsing algorithm in parse_os() In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-18-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- libxkutil/device_parsing.c | 120 ++++++++++++++++++++++++++++++++------------- 1 file changed, 87 insertions(+), 33 deletions(-) diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c index b244d8a..56c9d3d 100644 --- a/libxkutil/device_parsing.c +++ b/libxkutil/device_parsing.c @@ -2617,9 +2617,8 @@ static void cleanup_bootlist(char **blist, unsigned blist_ct) free(blist); } -static int parse_os(struct domain *dominfo, xmlNode *os) +static int parse_os(struct domain *dominfo) { - xmlNode *child; char **blist = NULL; unsigned bl_size = 0; char *arch = NULL; @@ -2631,39 +2630,93 @@ static int parse_os(struct domain *dominfo, xmlNode *os) char *boot = NULL; char *init = NULL; - for (child = os->children; child != NULL; child = child->next) { - if (XSTREQ(child->name, "type")) { - STRPROP(dominfo, os_info.pv.type, child); - arch = get_attr_value(child, "arch"); - machine = get_attr_value(child, "machine"); - } else if (XSTREQ(child->name, "kernel")) - kernel = get_node_content(child); - else if (XSTREQ(child->name, "initrd")) - initrd = get_node_content(child); - else if (XSTREQ(child->name, "cmdline")) - cmdline = get_node_content(child); - else if (XSTREQ(child->name, "loader")) - loader = get_node_content(child); - else if (XSTREQ(child->name, "boot")) { - char **tmp_list = NULL; - - tmp_list = (char **)realloc(blist, - (bl_size + 1) * - sizeof(char *)); - if (tmp_list == NULL) { - // Nothing you can do. Just go on. - CU_DEBUG("Could not alloc space for " - "boot device"); - continue; - } - blist = tmp_list; + CU_DEBUG("Enter parse_os()"); + + dominfo->os_info.pv.type = fetch_from_others(&dominfo->others, + 0, + "type", + TYPE_NODE, + 0, + "os"); + if (dominfo->os_info.pv.type) { + arch = fetch_from_others(&dominfo->others, + -1, + "arch", + TYPE_PROP, + -1, + "type"); + + machine = fetch_from_others(&dominfo->others, + -1, + "machine", + TYPE_PROP, + -1, + "type"); + } + + dominfo->os_info.pv.kernel = fetch_from_others(&dominfo->others, + -1, + "kernel", + TYPE_NODE, + -1, + "os"); + + dominfo->os_info.pv.initrd = fetch_from_others(&dominfo->others, + -1, + "initrd", + TYPE_NODE, + -1, + "os"); + + dominfo->os_info.pv.cmdline = fetch_from_others(&dominfo->others, + -1, + "cmdline", + TYPE_NODE, + -1, + "os"); + + dominfo->os_info.fv.loader = fetch_from_others(&dominfo->others, + -1, + "loader", + TYPE_NODE, + -1, + "os"); + + if (seek_in_others(&dominfo->others, + -1, + "boot", + TYPE_NODE, + -1, + "os")) { + char **tmp_list = NULL; + + tmp_list = (char **)realloc(blist, + (bl_size + 1) * sizeof(char *)); + + if (tmp_list == NULL) { + /* Nothing you can do. Just go on. */ + CU_DEBUG("Could not alloc space for " + "boot device"); + } else { + blist = tmp_list; - blist[bl_size] = get_attr_value(child, "dev"); + blist[bl_size] = fetch_from_others(&dominfo->others, + -1, + "dev", + TYPE_PROP, + -1, + "boot"); bl_size++; - } else if (XSTREQ(child->name, "init")) - init = get_node_content(child); + } } + dominfo->os_info.lxc.init = fetch_from_others(&dominfo->others, + -1, + "init", + TYPE_NODE, + -1, + "os"); + if ((STREQC(dominfo->os_info.fv.type, "hvm")) && (STREQC(dominfo->typestr, "xen"))) dominfo->type = DOMAIN_XENFV; @@ -2673,7 +2726,8 @@ static int parse_os(struct domain *dominfo, xmlNode *os) dominfo->type = DOMAIN_QEMU; else if (STREQC(dominfo->typestr, "lxc")) dominfo->type = DOMAIN_LXC; - else if (STREQC(dominfo->os_info.pv.type, "linux")) + else if (dominfo->os_info.pv.type && + STREQC(dominfo->os_info.pv.type, "linux")) dominfo->type = DOMAIN_XENPV; else dominfo->type = -1; @@ -2833,7 +2887,7 @@ static int parse_domain(xmlNodeSet *nsv, struct domain *dominfo) TYPE_NODE, -1, (char *)nodes[0]->name)) { - /* parse_os(); */ + parse_os(dominfo); } action = fetch_from_others(&dominfo->others, -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:24:09 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:24:09 -0500 Subject: [Libvirt-cim] [PATCH 20/20] Add type CIM_RES_TYPE_DELETED and modify type as it after resource_del In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <1384475049-11435-21-git-send-email-jferlan@redhat.com> From: Xu Wang Signed-off-by: Xu Wang --- src/Virt_VirtualSystemManagementService.c | 2 +- src/svpc_types.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/src/Virt_VirtualSystemManagementService.c b/src/Virt_VirtualSystemManagementService.c index 5c7238f..e1f7640 100644 --- a/src/Virt_VirtualSystemManagementService.c +++ b/src/Virt_VirtualSystemManagementService.c @@ -3032,7 +3032,7 @@ static CMPIStatus resource_del(struct domain *dominfo, CLASSNAME(op)); } - dev->type = CIM_RES_TYPE_UNKNOWN; + dev->type = CIM_RES_TYPE_DELETED; break; } diff --git a/src/svpc_types.h b/src/svpc_types.h index 4928742..e70c16c 100644 --- a/src/svpc_types.h +++ b/src/svpc_types.h @@ -32,6 +32,7 @@ #define CIM_RES_TYPE_DISK 17 #define CIM_RES_TYPE_GRAPHICS 24 #define CIM_RES_TYPE_INPUT 13 +#define CIM_RES_TYPE_DELETED 999 #define CIM_RES_TYPE_UNKNOWN 1000 #define CIM_RES_TYPE_UNKDEV 1001 #define CIM_RES_TYPE_IMAGE 32768 -- 1.8.3.1 From jferlan at redhat.com Fri Nov 15 00:29:09 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 14 Nov 2013 19:29:09 -0500 Subject: [Libvirt-cim] [RESEND PATCH 0/3] libvirt-cim: Fix provider registration In-Reply-To: <5284A5C0.8090101@linux.vnet.ibm.com> References: <1383642213-10260-1-git-send-email-mihajlov@linux.vnet.ibm.com> <528144F6.6030907@redhat.com> <52825ACA.4000704@de.ibm.com> <5282668C.8080000@redhat.com> <5284A5C0.8090101@linux.vnet.ibm.com> Message-ID: <52856AD5.6030006@redhat.com> On 11/14/2013 05:28 AM, Daniel Hansel wrote: > Hi John, > > after investigating in this problem we have seen that with tog-pegasus version 2.12.1-5 the namespace for base classes was changed from root/PG_Interop to root/interop. > tog-pegasus can be build using an option that changes this namespace to one of these 2 namespaces. > As far as we could see RedHat is using this option (and the new namespace) in their build of tog-pegasus. > > The cim providers that should be installed have to use the correct namespace to find the right classes. > This would lead to an incompatibility of cim providers AND clients. > > Therefore the pegasus community does NOT recommend the change to the new namespace until a backwards compatible way is implemented in tog-pegasus. > > Kind regards, > Daniel > So just to be sure - if I'm reading this right, tog-pegasus 2.12.1-5 and beyond have broken libvirt-cim? Do the tog-pegasus developers know this? Is there a bug to track? Was there some expectation that libvirt-cim "change" to using root/interop? I guess the hard part for me is that anyone getting the latest updates to f19 will run into this. It doesn't matter for RHEL6 since 6.5 at least since it's only up to 2.12.0-3. Tks, John From fiuczy at linux.vnet.ibm.com Fri Nov 15 12:17:28 2013 From: fiuczy at linux.vnet.ibm.com (Boris Fiuczynski) Date: Fri, 15 Nov 2013 13:17:28 +0100 Subject: [Libvirt-cim] [RESEND PATCH 0/3] libvirt-cim: Fix provider registration In-Reply-To: <52856AD5.6030006@redhat.com> References: <1383642213-10260-1-git-send-email-mihajlov@linux.vnet.ibm.com> <528144F6.6030907@redhat.com> <52825ACA.4000704@de.ibm.com> <5282668C.8080000@redhat.com> <5284A5C0.8090101@linux.vnet.ibm.com> <52856AD5.6030006@redhat.com> Message-ID: <528610D8.9060403@linux.vnet.ibm.com> On 11/15/2013 01:29 AM, John Ferlan wrote: > On 11/14/2013 05:28 AM, Daniel Hansel wrote: > >> Hi John, >> >> after investigating in this problem we have seen that with tog-pegasus version 2.12.1-5 the namespace for base classes was changed from root/PG_Interop to root/interop. >> tog-pegasus can be build using an option that changes this namespace to one of these 2 namespaces. >> As far as we could see RedHat is using this option (and the new namespace) in their build of tog-pegasus. >> >> The cim providers that should be installed have to use the correct namespace to find the right classes. >> This would lead to an incompatibility of cim providers AND clients. >> >> Therefore the pegasus community does NOT recommend the change to the new namespace until a backwards compatible way is implemented in tog-pegasus. >> >> Kind regards, >> Daniel >> > > So just to be sure - if I'm reading this right, tog-pegasus 2.12.1-5 and > beyond have broken libvirt-cim? > > Do the tog-pegasus developers know this? Is there a bug to track? > > Was there some expectation that libvirt-cim "change" to using > root/interop? I guess the hard part for me is that anyone getting the > latest updates to f19 will run into this. It doesn't matter for RHEL6 > since 6.5 at least since it's only up to 2.12.0-3. > > Tks, > > John > > _______________________________________________ > Libvirt-cim mailing list > Libvirt-cim at redhat.com > https://www.redhat.com/mailman/listinfo/libvirt-cim > John, looking a bit more at this I found that other provider libraries have been adjusted to the namespace change. I have put Vitezslav Crhonek on cc who provided the changes for the sblim-cmpi libraries. Here is an example (see changelog): http://s390pkgs.fedoraproject.org/koji/buildinfo?buildID=209208 Mit freundlichen Gr??en/Kind regards Boris Fiuczynski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martina K?deritz Gesch?ftsf?hrung: Dirk Wittkopp Sitz der Gesellschaft: B?blingen Registergericht: Amtsgericht Stuttgart, HRB 243294 From mihajlov at linux.vnet.ibm.com Fri Nov 15 16:00:16 2013 From: mihajlov at linux.vnet.ibm.com (Viktor Mihajlovski) Date: Fri, 15 Nov 2013 17:00:16 +0100 Subject: [Libvirt-cim] [PATCH 0/2] cimtest support for consoles In-Reply-To: <5282C622.9060501@redhat.com> References: <1381489788-21481-1-git-send-email-mihajlov@linux.vnet.ibm.com> <5282C622.9060501@redhat.com> Message-ID: <52864510.9080808@linux.vnet.ibm.com> On 11/13/2013 01:21 AM, John Ferlan wrote: > > Not sure how to approach this series. On the one hand things look good; > however, when run with Xu Wang's 48 patch series it seems there's > assumptions in one set of patches or the other. > > In particular, there's numerous failures with the libvirt message "Only > the first console can be a serial port", which means for some reason > there's more than one console in the guest configured to use the serial > line. Since with the read-only part of your rework of Xu Wang's series, the results remain good, I suggest to try to find out why the assumptions of cimtest don't match the write-back result (once this gets enabled) step by step. This could result in changes to either of the pending libvirt-cim or cimtest patches... -- Mit freundlichen Gr??en/Kind Regards Viktor Mihajlovski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martina K?deritz Gesch?ftsf?hrung: Dirk Wittkopp Sitz der Gesellschaft: B?blingen Registergericht: Amtsgericht Stuttgart, HRB 243294 From mihajlov at linux.vnet.ibm.com Fri Nov 15 16:04:10 2013 From: mihajlov at linux.vnet.ibm.com (Viktor Mihajlovski) Date: Fri, 15 Nov 2013 17:04:10 +0100 Subject: [Libvirt-cim] [PATCH 00/20] REWORK/PARIAL: Changes to solve unsupported tag issue In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <528645FA.7020305@linux.vnet.ibm.com> On 11/15/2013 01:23 AM, John Ferlan wrote: > This is a *partial rework* of Xu Wang's patches sent last month: > > https://www.redhat.com/archives/libvirt-cim/2013-October/msg00081.html > > Although not the complete set of changes - it's a good stopping point > insomuch as it handles the "others" parsing. If this looks good, I can > push it, then work through the changes to write the xml. > > I have run all the changes through cimtest - even with the patches on the > list from Viktor. No new issues are found. > > wow, hats off to that :-) However I need to report an issue running on s390, cimprovagt core dumps, so I need to investigate further and ask to please hold off until I figure out the reason. Thanks! -- Mit freundlichen Gr??en/Kind Regards Viktor Mihajlovski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martina K?deritz Gesch?ftsf?hrung: Dirk Wittkopp Sitz der Gesellschaft: B?blingen Registergericht: Amtsgericht Stuttgart, HRB 243294 From jferlan at redhat.com Fri Nov 15 18:41:18 2013 From: jferlan at redhat.com (John Ferlan) Date: Fri, 15 Nov 2013 13:41:18 -0500 Subject: [Libvirt-cim] [PATCH 00/20] REWORK/PARIAL: Changes to solve unsupported tag issue In-Reply-To: <528645FA.7020305@linux.vnet.ibm.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> <528645FA.7020305@linux.vnet.ibm.com> Message-ID: <52866ACE.8050803@redhat.com> On 11/15/2013 11:04 AM, Viktor Mihajlovski wrote: > On 11/15/2013 01:23 AM, John Ferlan wrote: >> This is a *partial rework* of Xu Wang's patches sent last month: >> >> https://www.redhat.com/archives/libvirt-cim/2013-October/msg00081.html >> >> Although not the complete set of changes - it's a good stopping point >> insomuch as it handles the "others" parsing. If this looks good, I can >> push it, then work through the changes to write the xml. >> >> I have run all the changes through cimtest - even with the patches on the >> list from Viktor. No new issues are found. >> >> > wow, hats off to that :-) thanks - it was a way to review too as the series was just so large... > > However I need to report an issue running on s390, cimprovagt > core dumps, so I need to investigate further and ask to > please hold off until I figure out the reason. Thanks! > I found investigating cimprovagt to be very painful, hence the reason why I redid 1-15 a bit. I'd suggest trying to apply 1-3 first - make sure they work. Then 4-13 to make sure they work. Then go slower on 14-20. I found that 14/15 were the most problematic... 16-18 were mechanical. 19 seemed to be harmless; however, who knows. The issue with 14/15 was that CIM_RES_TYPE_UNKNOWN already existed and was in use for other things - that's why I added UNKDEV, although I could have messed the numbering scheme up. I know there are specific rules about what numbers can be used, but I'm not sure of the details... John From jferlan at redhat.com Fri Nov 15 20:14:46 2013 From: jferlan at redhat.com (John Ferlan) Date: Fri, 15 Nov 2013 15:14:46 -0500 Subject: [Libvirt-cim] [PATCH 1/2] cimtest: Adding support for ConsoleRASD In-Reply-To: <1381489788-21481-2-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1381489788-21481-1-git-send-email-mihajlov@linux.vnet.ibm.com> <1381489788-21481-2-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <528680B6.7020707@redhat.com> On 10/11/2013 07:09 AM, Viktor Mihajlovski wrote: > The test domains are now defined with a ConsoleRASD if the > tested libvirt-cim does support those. A few testcases had > to be adjusted to properly handle the newly encountered instances > and associations. > > Signed-off-by: Viktor Mihajlovski <...snip...> > diff --git a/suites/libvirt-cim/lib/XenKvmLib/vxml.py b/suites/libvirt-cim/lib/XenKvmLib/vxml.py > index 74c4f23..9489a5c 100644 > --- a/suites/libvirt-cim/lib/XenKvmLib/vxml.py > +++ b/suites/libvirt-cim/lib/XenKvmLib/vxml.py > @@ -50,9 +50,11 @@ from CimTest.ReturnCodes import SKIP, PASS, FAIL > from XenKvmLib.classes import virt_types, get_typed_class > from XenKvmLib.enumclass import GetInstance > from XenKvmLib.const import get_provider_version > +from XenKvmLib.xm_virt_util import host_cpu_model > > vsms_graphics_sup = 763 > vsms_inputdev_sup = 771 > +vsms_console_sup = 1276 > > class XMLClass: > xml_string = "" > @@ -598,7 +600,8 @@ class VirtCIM: > def __init__(self, virt, dom_name, uuid, pae, acpi, apic, disk_dev, > disk_source, net_type, net_name, net_mac, vcpus, mem, > mem_allocunits, emu_type, grstype, ip, > - is_ipv6_only, port_num, kmap, irstype, btype, vnc_passwd): > + is_ipv6_only, port_num, kmap, irstype, btype, vnc_passwd, > + con_source_type, con_target_type): > self.virt = virt > self.domain_name = dom_name > self.err_rc = None > @@ -623,6 +626,9 @@ class VirtCIM: > ipv6_flag=is_ipv6_only, > lport=port_num, keymap=kmap, > vnc_passwd=vnc_passwd) > + self.casd = vsms.get_casd_class(virt)(name=dom_name, > + source_type=con_source_type, > + target_type=con_target_type) > self.masd = vsms.get_masd_class(virt)(megabytes=mem, > mallocunits=mem_allocunits, > name=dom_name) > @@ -655,9 +661,14 @@ class VirtCIM: > res_settings.append(str(self.gasd)) > > if curr_cim_rev >= vsms_inputdev_sup: > - if self.iasd is not None: > + if self.iasd is not None and \ > + not str(host_cpu_model(ip, self.virt)).startswith('s390'): > res_settings.append(str(self.iasd)) > > + if curr_cim_rev >= vsms_console_sup: > + if ref_conf is None and self.casd is not None: > + res_settings.append(str(self.casd)) > + I've been working on applying the "next set" of Xu Wang's fixes and found that by just adding the four core XenKvmLib/*.py files to a test branch I could core FilterList 03_create.py test rather reliably until I commented out the above 3 lines. The first change applied that caused the failure was the writing of the console xml (#22 in the series). The core was because for some reason the dumping of the xml didn't find the "" tag and thus the call to "filterref = test.libvirt_applied_filter_lists(domain_name)[0]" would have an index out of range exception. Also a number of the VirtualSystemManagementService and VSSD tests failed with the "Only the first console can be a serial port..." message. Unfortunately I missed the notification that the system I had borrowed had timed out its lease and thus I've lost usage for a bit. However, I do know without this particular patch, the other changes did run through cimtest. I'm going to try and revive the ability to run on my own f19 system, I have to find a tog-pegasus that works for me.... John From fiuczy at linux.vnet.ibm.com Mon Nov 18 14:59:36 2013 From: fiuczy at linux.vnet.ibm.com (Boris Fiuczynski) Date: Mon, 18 Nov 2013 15:59:36 +0100 Subject: [Libvirt-cim] [PATCH 00/20] REWORK/PARIAL: Changes to solve unsupported tag issue In-Reply-To: <1384475049-11435-1-git-send-email-jferlan@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> Message-ID: <528A2B58.5010907@linux.vnet.ibm.com> John and Xu Wang, here are a few general observations from side: 1) I agree that it makes sense to preserve the unknown xml "entities" even so it can create complex scenarios and even new kinds of errors if unknown entities depend on known entities which get modified making them unusable for the unknown entities. This error would probably be the reversal of what is currently the problem when unknown entities simply disappear. 2) The implementation of putting hierarchical data (xml elements and properties) into a single "flat" list and storing the hierarchical dependencies with strings worries me. Wouldn't it make things easier to create the hierarchy in the others structure (it also would remove the need for the ids I guess!). The others structure is stored on the internal data object that represents a defined known xml entity. This defined known xml entity can itself be the parent for unknown properties (xml properites) and/or the parent for unknown sub entities (xml elements) where all these entities at the end of the "data parsing" would be flagged ACTIVE and the unknown ones INACTIVE. /* This could actually be just other_node */ struct others { struct other_node *known_node; /* this is the root representing the defined known xml element */ }; struct other_node { char *name; char *value; struct other_prop *properties; /* list of the nodes properties */ struct other_node *children; /* list with the direct sets of children data */ enum { ACTIVE, INACTIVE } status; }; struct other_prop { char *name; char *value; struct other_prop *properties; enum { ACTIVE, INACTIVE } status; }; Probably the above data structures could be streamlined even more when one starts to use these structures writing the code. The structures are very close to the xml and recursive methods could be used for writing the xml out. References to the other_node and other_prop structures could be made available for writing and reading data from them via methods. e.g. writing back the console (just a snippet from xmlgen.c) - console = xmlNewChild(root, NULL, BAD_CAST "console", NULL); - if (console == NULL) + other_node *console = NULL; + console = add_node_to_others(cdev->others->known_node, + "console", + NULL, /* text value of the xml element */ + "devices"); + + if (console == NULL) { + CU_DEBUG("Add tag failed."); return XML_ERROR; + } - xmlNewProp(console, BAD_CAST "type", - BAD_CAST - chardev_source_type_IDToStr(cdev->source_type)); + other_prop *cprop = NULL; + cprop = add_prop_to_others(console, + "type", + chardev_source_type_IDToStr( + cdev->source_type)); As you can see it is much closer to the known pluming as it used to be directly with xml and also hides the list internals (name & ids) used for referencing. I know that this would cause a rather large rework but I think that the usability would be much enhanced for everyone writing/fixing the provider code and overall would improve code stability in the long run. Please use this as an idea for improvement. Also note: The above is just email written code for reference and not thought to be bug free. :-) Mit freundlichen Gr??en/Kind regards Boris Fiuczynski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martina K?deritz Gesch?ftsf?hrung: Dirk Wittkopp Sitz der Gesellschaft: B?blingen Registergericht: Amtsgericht Stuttgart, HRB 243294 On 11/15/2013 01:23 AM, John Ferlan wrote: > This is a *partial rework* of Xu Wang's patches sent last month: > > https://www.redhat.com/archives/libvirt-cim/2013-October/msg00081.html > > Although not the complete set of changes - it's a good stopping point > insomuch as it handles the "others" parsing. If this looks good, I can > push it, then work through the changes to write the xml. > > I have run all the changes through cimtest - even with the patches on the > list from Viktor. No new issues are found. > > Changes to the original patches > > 1. I rebased to top of tree as of today (11/14/13) > > 2. I reworked each of the patches to only add the 'others' to the > particular *_device structure. This was done to be able to show > the progression and to ensure I didn't forget something > > 2a. This found that there needed to be a cleanup_vcpu_device() and > cleanup_mem_device() since both added "others" to their structure > but were never cleaned up during cleanup_virt_device() > 2b. I did not see a need for "others" in "vnc_device" and "sdl_device" > 2c. I added a cleanup_others() call in _get_dominfo() > > 3. I added fetch_device_address_from_others() to replace parse_device_address(). > This bridges the "gap" bewteen the parsing of the
tag that was > recently added and the 'others' parsing code which didn't handle it. > All I did was traverse the others list looking for parent name and id > match for TYPE_PROP entries. Essentially anything inside of > us then copied in to name/value structure managed by devaddr. Once the last > caller of parse_device_address() was removed, that function went away > > 4. I changed the type for unknown from CIM_RES_TYPE_UNKNOWN to > CIM_RES_TYPE_UNKDEV. Turns out the former is used in other contexts > and if/when cleanup_virt_device() was called from one of those contexts > the result was a core in cimprovagt. This was seen in the cimtest > for VirtualSystemManagementService 16_removeresource.py. > > 4. Various "bug fixes" based on a coverity run and what I saw in code > review from Boris. > - Original code had a "ddev->disk_type == DISK_PHY;" > - There was a "if (node->name == NULL)" check in parse_graphics_device > after "else if (STREQC(gdev->type, "pty")) {" which coverity flagged > as unnecessary since earlier code assumed node->name != NULL > - In parse_os() there was a "STREQC(dominfo->os_info.pv.type, "linux")" > which needed a "dominfo->os_info.pv.type &&" prior to it since the value > could have been NULL according to a check earlier in the routine > - I added the "free(dev->name);" to cleanup_unknown_device() > - Checks for others->{name|parent_name|value} were removed. Since the > structures are calloc()'d and free(NULL) does nothing, this is OK. > > Xu Wang (20): > Add others member for saving unsupported tag and unknown device > Add others and unknown_device clean up > Add basic operations for reading data from xml node > Fix xml parsing algorithm for parse_fs_device() > Fix xml parsing algorithm for parse_block_device() > Fix xml parsing algorithm for parse_vsi_device() > Fix xml parsing algorithm for parse_net_device() > Fix xml parsing algorithm for parse_vcpu_device() > Fix xml parsing algorithm for parse_emu_device() > Fix xml parsing algorithm for parse_mem_device() > Fix xml parsing algorithm for parse_console_device() > Fix xml parsing algorithm for parse_graphics_device() > Fix xml parsing algorithm for parse_input_device() > Add parse_unknown_device() > Add parse_devices() for unknown type in get_dominfo_from_xml() > Fix xml parsing algorithm in parse_domain() > Fix xml parsing algorithm in parse_os() > Fix xml parsing algorithm in parse_features() > Add dup function for device copy > Add type CIM_RES_TYPE_DELETED and modify type as it after resource_del > > libxkutil/device_parsing.c | 2055 ++++++++++++++++++++++++----- > libxkutil/device_parsing.h | 58 + > src/Virt_VirtualSystemManagementService.c | 2 +- > src/svpc_types.h | 2 + > 4 files changed, 1774 insertions(+), 343 deletions(-) > From jferlan at redhat.com Mon Nov 18 22:12:41 2013 From: jferlan at redhat.com (John Ferlan) Date: Mon, 18 Nov 2013 17:12:41 -0500 Subject: [Libvirt-cim] [RESEND PATCH 0/3] libvirt-cim: Fix provider registration In-Reply-To: <528610D8.9060403@linux.vnet.ibm.com> References: <1383642213-10260-1-git-send-email-mihajlov@linux.vnet.ibm.com> <528144F6.6030907@redhat.com> <52825ACA.4000704@de.ibm.com> <5282668C.8080000@redhat.com> <5284A5C0.8090101@linux.vnet.ibm.com> <52856AD5.6030006@redhat.com> <528610D8.9060403@linux.vnet.ibm.com> Message-ID: <528A90D9.4070602@redhat.com> > > John, > looking a bit more at this I found that other provider libraries have > been adjusted to the namespace change. I have put Vitezslav Crhonek on > cc who provided the changes for the sblim-cmpi libraries. > Here is an example (see changelog): > http://s390pkgs.fedoraproject.org/koji/buildinfo?buildID=209208 > I took a stab at this considering it didn't seem "too difficult" - the main differences being 1. Using "root/interop" instead of "root/PG_InterOp" in provider-register.sh (if the installed cimserver >= 2.12.1) 2. Adjusting libvirt-cim.spec.in similarly with checks for version and dealing the PG_InterOp (although I'm less convinced it makes much of a difference other than perhaps cleaning something up). Anyway, a subsequent run of cimtest was "mostly" successful - I had some issues with FilterList again. It seems the code uses "InstanceID" in order to save the UUID from the filter and in my new environment that doesn't appear to be defined. Whether that's a CIM thing or not I'm still trying to figure out. So before I go digging too deep and taking too long to figure out what's up - does it seem plausible that by just changing from PG_InterOp to interop that we'd have to perhaps adjust the schema? Perhaps create a 'uuid' field? Could perhaps InstanceID be something we cannot write to? (See src/Virt_FilterList.c). Tks, John From jferlan at redhat.com Tue Nov 19 13:25:43 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 19 Nov 2013 08:25:43 -0500 Subject: [Libvirt-cim] [PATCH] live.full_hostname: Adjust mechanism to get FQDN Message-ID: <1384867543-8939-1-git-send-email-jferlan@redhat.com> Using just socket.gethostbyaddr(socket.gethostname())[0] caused some issues recently in one of my DHCP testing environments. Breaking down the calls showed the following: >>> socket.gethostname() 'dhcp-186-211.bos.redhat.com' >>> socket.gethostbyname("dhcp-186-211.bos.redhat.com") '10.16.186.211' >>> socket.gethostbyaddr(socket.gethostname())[0] Traceback (most recent call last): File "", line 1, in socket.herror: [Errno 1] Unknown host While just a socket.gethostname() could have worked, using the socket.getfqdn() seemed to be safer just in case. --- lib/VirtLib/live.py | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/lib/VirtLib/live.py b/lib/VirtLib/live.py index c929e71..6a10474 100644 --- a/lib/VirtLib/live.py +++ b/lib/VirtLib/live.py @@ -100,6 +100,9 @@ def hostname(server): return out def full_hostname(server): - """To return the fully qualifiec domain name(FQDN) of the system""" + """To return the fully qualified domain name(FQDN) of the system""" - return socket.gethostbyaddr(socket.gethostname())[0] + if socket.getfqdn().find('.') >= 0: + return socket.getfqdn() + else: + return socket.gethostbyaddr(socket.gethostname())[0] -- 1.8.3.1 From jferlan at redhat.com Tue Nov 19 21:02:54 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 19 Nov 2013 16:02:54 -0500 Subject: [Libvirt-cim] [PATCH 2/5] libxkutil: Support for device addresses In-Reply-To: <1381764584-18025-3-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1381764584-18025-1-git-send-email-mihajlov@linux.vnet.ibm.com> <1381764584-18025-3-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <528BD1FE.6070306@redhat.com> On 10/14/2013 11:29 AM, Viktor Mihajlovski wrote: > New data type for device addresses based on the generic > "address" element of the libvirt domain XML. > Contains XML parsing and generation code for device addresses. > > Signed-off-by: Viktor Mihajlovski > --- > libxkutil/device_parsing.c | 113 ++++++++++++++++++++++++++++++++++++++++++++ > libxkutil/device_parsing.h | 11 +++++ > libxkutil/xmlgen.c | 28 +++++++++++ > 3 files changed, 152 insertions(+) > Although this is already pushed, while I was working through Xu's patch set I think I found a missed case. The parsing code handles "parse_fs_device", "parse_block_device", and "parse_net_device"; however, the generation code does "disk_block_xml", "disk_file_xml", and "net_xml" missing out on "disk_fs_xml". The "parse_block_device" handles both "source=file" and "source=dev", while "disk_xml" checks for DISK_PHY or DISK_FILE before parceling out to the corresponding subroutine. John > diff --git a/libxkutil/device_parsing.c b/libxkutil/device_parsing.c > index 076bec0..56e39c7 100644 > --- a/libxkutil/device_parsing.c > +++ b/libxkutil/device_parsing.c > @@ -63,6 +63,22 @@ > /* Device parse function */ > typedef int (*dev_parse_func_t)(xmlNode *, struct virt_device **); > > +static void cleanup_device_address(struct device_address *addr) > +{ > + int i; > + if (addr == NULL) > + return; > + > + for (i = 0; i < addr->ct; i++) { > + free(addr->key[i]); > + free(addr->value[i]); > + } > + > + free(addr->key); > + free(addr->value); > + addr->ct = 0; > +} > + > static void cleanup_disk_device(struct disk_device *dev) > { > if (dev == NULL) > @@ -77,6 +93,7 @@ static void cleanup_disk_device(struct disk_device *dev) > free(dev->virtual_dev); > free(dev->bus_type); > free(dev->access_mode); > + cleanup_device_address(&dev->address); > } > > static void cleanup_vsi_device(struct vsi_device *dev) > @@ -107,6 +124,7 @@ static void cleanup_net_device(struct net_device *dev) > free(dev->filter_ref); > free(dev->poolid); > cleanup_vsi_device(&dev->vsi); > + cleanup_device_address(&dev->address); > } > > static void cleanup_emu_device(struct emu_device *dev) > @@ -351,6 +369,67 @@ char *get_node_content(xmlNode *node) > return buf; > } > > +int add_device_address_property(struct device_address *devaddr, > + const char *key, > + const char *value) > +{ > + char *k = NULL; > + char *v = NULL; > + char **list = NULL; > + > + if (key != NULL && value != NULL) { > + k = strdup(key); > + v = strdup(value); > + if (k == NULL || v == NULL) > + goto err; > + > + list = realloc(devaddr->key, sizeof(char*) * (devaddr->ct+1)); > + if (list == NULL) > + goto err; > + devaddr->key = list; > + > + list = realloc(devaddr->value, sizeof(char*) * (devaddr->ct+1)); > + if (list == NULL) > + goto err; > + devaddr->value = list; > + > + devaddr->key[devaddr->ct] = k; > + devaddr->value[devaddr->ct] = v; > + devaddr->ct += 1; > + return 1; > + } > + > + err: > + free(k); > + free(v); > + free(list); > + return 0; > +} > + > + > +static int parse_device_address(xmlNode *anode, struct device_address *devaddr) > +{ > + xmlAttr *attr = NULL; > + char *name = NULL; > + char *value = NULL; > + > + for (attr = anode->properties; attr != NULL; attr = attr->next) { > + name = (char*) attr->name; > + value = get_attr_value(anode, name); > + if (!add_device_address_property(devaddr, name, value)) > + goto err; > + free(value); > + } > + > + return 1; > + > + err: > + cleanup_device_address(devaddr); > + free(value); > + > + return 0; > +} > + > static int parse_fs_device(xmlNode *dnode, struct virt_device **vdevs) > { > struct virt_device *vdev = NULL; > @@ -386,6 +465,8 @@ static int parse_fs_device(xmlNode *dnode, struct virt_device **vdevs) > } > } else if (XSTREQ(child->name, "driver")) { > ddev->driver_type = get_attr_value(child, "type"); > + } else if (XSTREQ(child->name, "address")) { > + parse_device_address(child, &ddev->address); > } > } > > @@ -459,6 +540,8 @@ static int parse_block_device(xmlNode *dnode, struct virt_device **vdevs) > ddev->readonly = true; > } else if (XSTREQ(child->name, "shareable")) { > ddev->shareable = true; > + } else if (XSTREQ(child->name, "address")) { > + parse_device_address(child, &ddev->address); > } > } > > @@ -598,6 +681,8 @@ static int parse_net_device(xmlNode *inode, struct virt_device **vdevs) > ndev->filter_ref = get_attr_value(child, "filter"); > } else if (XSTREQ(child->name, "virtualport")) { > parse_vsi_device(child, ndev); > + } else if (XSTREQ(child->name, "address")) { > + parse_device_address(child, &ndev->address); > #if LIBVIR_VERSION_NUMBER >= 9000 > } else if (XSTREQ(child->name, "bandwidth")) { > /* Network QoS bandwidth support */ > @@ -1167,6 +1252,32 @@ static int parse_devices(const char *xml, struct virt_device **_list, int type) > return count; > } > > +static void duplicate_device_address(struct device_address *to, const struct device_address *from) > +{ > + int i; > + > + if (from == NULL || to == NULL || from->ct == 0) > + return; > + > + to->ct = from->ct; > + to->key = calloc(from->ct, sizeof(char*)); > + to->value = calloc(from->ct, sizeof(char*)); > + if (to->key == NULL || to->value == NULL) > + goto err; > + > + for (i = 0; i < from->ct; i++) { > + to->key[i] = strdup(from->key[i]); > + to->value[i] = strdup(from->value[i]); > + if (to->key[i] == NULL || to->value[i] == NULL) > + goto err; > + } > + > + return; > + > + err: > + cleanup_device_address(to); > +} > + > struct virt_device *virt_device_dup(struct virt_device *_dev) > { > struct virt_device *dev; > @@ -1196,6 +1307,7 @@ struct virt_device *virt_device_dup(struct virt_device *_dev) > DUP_FIELD(dev, _dev, dev.net.vsi.profile_id); > dev->dev.net.reservation = _dev->dev.net.reservation; > dev->dev.net.limit = _dev->dev.net.limit; > + duplicate_device_address(&dev->dev.net.address, &_dev->dev.net.address); > } else if (dev->type == CIM_RES_TYPE_DISK) { > DUP_FIELD(dev, _dev, dev.disk.type); > DUP_FIELD(dev, _dev, dev.disk.device); > @@ -1209,6 +1321,7 @@ struct virt_device *virt_device_dup(struct virt_device *_dev) > dev->dev.disk.disk_type = _dev->dev.disk.disk_type; > dev->dev.disk.readonly = _dev->dev.disk.readonly; > dev->dev.disk.shareable = _dev->dev.disk.shareable; > + duplicate_device_address(&dev->dev.disk.address, &_dev->dev.disk.address); > } else if (dev->type == CIM_RES_TYPE_MEM) { > dev->dev.mem.size = _dev->dev.mem.size; > dev->dev.mem.maxsize = _dev->dev.mem.maxsize; > diff --git a/libxkutil/device_parsing.h b/libxkutil/device_parsing.h > index 4beac5c..92427c1 100644 > --- a/libxkutil/device_parsing.h > +++ b/libxkutil/device_parsing.h > @@ -33,6 +33,12 @@ > > #include "../src/svpc_types.h" > > +struct device_address { > + uint32_t ct; > + char **key; > + char **value; > +}; > + > struct vsi_device { > char *vsi_type; > char *manager_id; > @@ -56,6 +62,7 @@ struct disk_device { > char *bus_type; > char *cache; > char *access_mode; /* access modes for DISK_FS (filesystem) type */ > + struct device_address address; > }; > > struct net_device { > @@ -70,6 +77,7 @@ struct net_device { > uint64_t reservation; > uint64_t limit; > struct vsi_device vsi; > + struct device_address address; > }; > > struct mem_device { > @@ -257,6 +265,9 @@ int get_devices(virDomainPtr dom, struct virt_device **list, int type, > void cleanup_virt_device(struct virt_device *dev); > void cleanup_virt_devices(struct virt_device **devs, int count); > > +int add_device_address_property(struct device_address *devaddr, > + const char *key, const char *value); > + > char *get_node_content(xmlNode *node); > char *get_attr_value(xmlNode *node, char *attrname); > > diff --git a/libxkutil/xmlgen.c b/libxkutil/xmlgen.c > index 7e8801d..40e2905 100644 > --- a/libxkutil/xmlgen.c > +++ b/libxkutil/xmlgen.c > @@ -178,6 +178,26 @@ static const char *console_xml(xmlNodePtr root, struct domain *dominfo) > BAD_CAST cdev->target_type); > } > } > + > + return NULL; > +} > + > +static char *device_address_xml(xmlNodePtr root, struct device_address *addr) > +{ > + int i; > + xmlNodePtr address; > + > + if (addr == NULL || addr->ct == 0) > + return NULL; > + > + address = xmlNewChild(root, NULL, BAD_CAST "address", NULL); > + if (address == NULL) > + return XML_ERROR; > + > + for (i = 0; i < addr->ct; i++) { > + xmlNewProp(address, BAD_CAST addr->key[i] , BAD_CAST addr->value[i]); > + } > + > return NULL; > } > > @@ -225,6 +245,9 @@ static char *disk_block_xml(xmlNodePtr root, struct disk_device *dev) > if (dev->shareable) > xmlNewChild(disk, NULL, BAD_CAST "shareable", NULL); > > + if (dev->address.ct > 0) > + return device_address_xml(disk, &dev->address); > + > return NULL; > } > > @@ -279,6 +302,8 @@ static const char *disk_file_xml(xmlNodePtr root, struct disk_device *dev) > if (dev->shareable) > xmlNewChild(disk, NULL, BAD_CAST "shareable", NULL); > > + if (dev->address.ct > 0) > + return device_address_xml(disk, &dev->address); > > return NULL; > } > @@ -520,6 +545,9 @@ static const char *net_xml(xmlNodePtr root, struct domain *dominfo) > } > #endif > > + if (dev->dev.net.address.ct > 0) > + msg = device_address_xml(nic, &dev->dev.net.address); > + > if (STREQ(dev->dev.net.type, "network")) { > msg = set_net_source(nic, net, "network"); > } else if (STREQ(dev->dev.net.type, "bridge")) { > From jferlan at redhat.com Tue Nov 19 21:49:45 2013 From: jferlan at redhat.com (John Ferlan) Date: Tue, 19 Nov 2013 16:49:45 -0500 Subject: [Libvirt-cim] [PATCH 00/20] REWORK/PARIAL: Changes to solve unsupported tag issue In-Reply-To: <528A2B58.5010907@linux.vnet.ibm.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> <528A2B58.5010907@linux.vnet.ibm.com> Message-ID: <528BDCF9.8010109@redhat.com> On 11/18/2013 09:59 AM, Boris Fiuczynski wrote: > John and Xu Wang, > here are a few general observations from side: First off - I tend to find myself agreeing with Boris here. I think the concept is important and necessary; however, I'm not convinced the implementation goes far enough. > 1) I agree that it makes sense to preserve the unknown xml "entities" > even so it can create complex scenarios and even new kinds of errors if > unknown entities depend on known entities which get modified making them > unusable for the unknown entities. This error would probably be the > reversal of what is currently the problem when unknown entities simply > disappear. Is there a more concrete example of "new kinds of errors if unknown entities depend on known entities which get modified making them unusable for the unknown entities" that can be given? Just for clarity. I've read that line a few times and I'm still not sure :-) > 2) The implementation of putting hierarchical data (xml elements and > properties) into a single "flat" list and storing the hierarchical > dependencies with strings worries me. Wouldn't it make things easier to > create the hierarchy in the others structure (it also would remove the > need for the ids I guess!). The primary reason why I went through the trouble of adjusting the commit was to learn a bit more about the code, but I also was trying to figure out a way to split up the submit into more manageable chunks... While going through the changes I found myself wondering what purpose the parent values were serving and why we kept passing -1 for ID values. When I went to integrate the "device address" changes into this new model I found myself beginning to think that perhaps the model used there to grab the property key/value pairs from the node generically was more suited for what you were trying to do. Since you're trying to protect against is someone changing/extending the XML "node" to add more "properties" which are then not restored when libvirt-cim goes to write out the XML, then why not take the next step and just handle the whole hierarchy? > The others structure is stored on the internal data object that > represents a defined known xml entity. This defined known xml entity can > itself be the parent for unknown properties (xml properites) and/or the > parent for unknown sub entities (xml elements) where all these entities > at the end of the "data parsing" would be flagged ACTIVE and the unknown > ones INACTIVE. > > /* This could actually be just other_node */ > struct others { > struct other_node *known_node; /* this is the root representing > the defined known xml element */ > }; > > struct other_node { > char *name; > char *value; > struct other_prop *properties; /* list of the nodes properties */ > struct other_node *children; /* list with the direct sets of > children data */ > enum { > ACTIVE, > INACTIVE > } status; > }; > > struct other_prop { > char *name; > char *value; > struct other_prop *properties; > enum { > ACTIVE, > INACTIVE > } status; > }; > > Probably the above data structures could be streamlined even more when > one starts to use these structures writing the code. The structures are > very close to the xml and recursive methods could be used for writing > the xml out. See the "device_address" structure and subsequent processing by parse_device_address() and add_device_address_property() for more opaque view on how to process the xml. If you take the concept and apply it to say "" - I think you'll be able to create one structure for dominfo that contains one instance of the read XML. Then using the active/inactive tagging you should be able to tell what's been read/used by the libvirt-cim code. I'd even go as far to say why not add a "CHANGED" status, but I'm not as clear on the lifecycle with respect to how the code the code that ends up writing things out works. My one concern with a single instance would be what happens if it changes without libvirt-cim's knowledge? Not sure its possible or supported. I think the way the code is written now there's perhaps more than one traversal of the tree, although without seeing in "in action" I cannot be sure. > > References to the other_node and other_prop structures could be made > available for writing and reading data from them via methods. > e.g. writing back the console (just a snippet from xmlgen.c) > > - console = xmlNewChild(root, NULL, BAD_CAST "console", > NULL); > - if (console == NULL) > + other_node *console = NULL; > + console = add_node_to_others(cdev->others->known_node, > + "console", > + NULL, /* text value of the > xml element */ > + "devices"); > + > + if (console == NULL) { > + CU_DEBUG("Add tag failed."); > return XML_ERROR; > + } > > - xmlNewProp(console, BAD_CAST "type", > - BAD_CAST > - > chardev_source_type_IDToStr(cdev->source_type)); > + other_prop *cprop = NULL; > + cprop = add_prop_to_others(console, > + "type", > + chardev_source_type_IDToStr( > + cdev->source_type)); > > As you can see it is much closer to the known pluming as it used to be > directly with xml and also hides the list internals (name & ids) used > for referencing. > > I know that this would cause a rather large rework but I think that the > usability would be much enhanced for everyone writing/fixing the > provider code and overall would improve code stability in the long run. > Please use this as an idea for improvement. I think as part of the rewrite creating macros to replace commonly cut-n-pasted code is a must. Currently there's numerous calls : + ddev->others = parse_data_to_others(ddev->others, + dnode, + 0, + BAD_CAST "devices"); + if (ddev->others == NULL) { + CU_DEBUG("parse xml failed."); + goto err; + } + or + if (seek_in_others(&ddev->others, + -1, + "source", + TYPE_NODE, + -1, + (char *)dnode->name)) { + ddev->source = fetch_from_others(&ddev->others, + -1, + "dir", + TYPE_PROP, + -1, + "source"); + + + if (ddev->source == NULL) { + CU_DEBUG("no source dir"); + goto err; + } + } There's macros that hide and do the error processing: GET_BASE_NODE(ddev, dnode, "devices", err); or ddev->source = GET_NODE_NODE(ddev, dnode, "source", err); if (ddev->source) { ddev->dir = GET_NODE_PROP(ddev->source, "dir", err); ddev->startupPolicy = GET_NODE_PROP(ddev, "startupPolicy", err); GET_NODE_NODE(ddev, ddev->source, "address", err); if (ddev->address) { } } The various macros would handle the CU_DEBUG (and any other error) processing. Similarly there would be PUT_BASE_NODE, PUT_NODE_NODE, PUT_NODE_PROP. The macros aren't completely thought out, but I would hope you see their value... > Also note: The above is just email written code for reference and not > thought to be bug free. :-) Hah - bug free code... If it were all bug free we'd be out of a job :-) I think we need to come up with an agreement on the architecture first (currently patches 1-3 and then what was the original 21/48). Those should be submitted first without any other patches and without necessarily being called except perhaps through xml_parse_test which should be used to prove that the "new" parsing and generation does no worse than the old (a 5th patch perhaps). Once those patches are in, then submit the patches that read/write domain level data... Then submit the patches that read/write domain/mem data... then domain/vcpu, etc. Also, rather than separate the xml parsing & generating, keep it together for each type. Doing things in smaller bunches will be easier to review and less prone to "other" changes causing issues. As an aside, I'm personally not a fan of the name 'others', but I don't have a better suggestion :-) I do ask that you make the testing component of this a priority. Being able to use xml_parse_test to validate is key. You may also want to add code to it in order to print out elements that are "inactive" - e.g. all the unknown stuff. Having visibility into what is not processed would be a good thing. John > > Mit freundlichen Gr??en/Kind regards > Boris Fiuczynski > > IBM Deutschland Research & Development GmbH > Vorsitzender des Aufsichtsrats: Martina K?deritz > Gesch?ftsf?hrung: Dirk Wittkopp > Sitz der Gesellschaft: B?blingen > Registergericht: Amtsgericht Stuttgart, HRB 243294 > > On 11/15/2013 01:23 AM, John Ferlan wrote: From cngesaint at gmail.com Wed Nov 20 08:36:56 2013 From: cngesaint at gmail.com (Xu Wang) Date: Wed, 20 Nov 2013 16:36:56 +0800 Subject: [Libvirt-cim] [PATCH 00/20] REWORK/PARIAL: Changes to solve unsupported tag issue In-Reply-To: <528BDCF9.8010109@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> <528A2B58.5010907@linux.vnet.ibm.com> <528BDCF9.8010109@redhat.com> Message-ID: <528C74A8.9010205@gmail.com> ? 2013/11/20 5:49, John Ferlan ??: > On 11/18/2013 09:59 AM, Boris Fiuczynski wrote: >> John and Xu Wang, >> here are a few general observations from side: > First off - I tend to find myself agreeing with Boris here. I think the > concept is important and necessary; however, I'm not convinced the > implementation goes far enough. > >> 1) I agree that it makes sense to preserve the unknown xml "entities" >> even so it can create complex scenarios and even new kinds of errors if >> unknown entities depend on known entities which get modified making them >> unusable for the unknown entities. This error would probably be the >> reversal of what is currently the problem when unknown entities simply >> disappear. > Is there a more concrete example of "new kinds of errors if unknown > entities depend on known entities which get modified making them > unusable for the unknown entities" that can be given? Just for clarity. > I've read that line a few times and I'm still not sure :-) > > >> 2) The implementation of putting hierarchical data (xml elements and >> properties) into a single "flat" list and storing the hierarchical >> dependencies with strings worries me. Wouldn't it make things easier to >> create the hierarchy in the others structure (it also would remove the >> need for the ids I guess!). > The primary reason why I went through the trouble of adjusting the > commit was to learn a bit more about the code, but I also was trying to > figure out a way to split up the submit into more manageable chunks... > While going through the changes I found myself wondering what purpose > the parent values were serving and why we kept passing -1 for ID values. > > When I went to integrate the "device address" changes into this new > model I found myself beginning to think that perhaps the model used > there to grab the property key/value pairs from the node generically was > more suited for what you were trying to do. > > Since you're trying to protect against is someone changing/extending the > XML "node" to add more "properties" which are then not restored when > libvirt-cim goes to write out the XML, then why not take the next step > and just handle the whole hierarchy? > > >> The others structure is stored on the internal data object that >> represents a defined known xml entity. This defined known xml entity can >> itself be the parent for unknown properties (xml properites) and/or the >> parent for unknown sub entities (xml elements) where all these entities >> at the end of the "data parsing" would be flagged ACTIVE and the unknown >> ones INACTIVE. >> >> /* This could actually be just other_node */ >> struct others { >> struct other_node *known_node; /* this is the root representing >> the defined known xml element */ >> }; >> >> struct other_node { >> char *name; >> char *value; >> struct other_prop *properties; /* list of the nodes properties */ >> struct other_node *children; /* list with the direct sets of >> children data */ >> enum { >> ACTIVE, >> INACTIVE >> } status; >> }; >> >> struct other_prop { >> char *name; >> char *value; >> struct other_prop *properties; >> enum { >> ACTIVE, >> INACTIVE >> } status; >> }; >> >> Probably the above data structures could be streamlined even more when >> one starts to use these structures writing the code. The structures are >> very close to the xml and recursive methods could be used for writing >> the xml out. > See the "device_address" structure and subsequent processing by > parse_device_address() and add_device_address_property() for more opaque > view on how to process the xml. > > If you take the concept and apply it to say "" - I think you'll > be able to create one structure for dominfo that contains one instance > of the read XML. Then using the active/inactive tagging you should be > able to tell what's been read/used by the libvirt-cim code. I'd even go > as far to say why not add a "CHANGED" status, but I'm not as clear on > the lifecycle with respect to how the code the code that ends up writing > things out works. My one concern with a single instance would be what > happens if it changes without libvirt-cim's knowledge? Not sure its > possible or supported. > > I think the way the code is written now there's perhaps more than one > traversal of the tree, although without seeing in "in action" I cannot > be sure. Dear John and Boris, I am considering pros/cons of suggestion from Boris. If the parsing process likes this, 1. Traversal the whole xml and save all data into data structure Boris describe above; 2. Calling ***_parse() functions in device_parsing.c fetch data from the data structure generated in step 1. 3. All kinds of resource operations. (Here I want to enhance the power of libvirt-cim after these patches merged. Some new name and value pairs could be passed from API and Management could be done. The work next step maybe more complex and I started to think about it) 4. Restore xml after operation, ***_xml() restore all data in the virt_devices fetched from data structure 5. Based on data structure restored, new xml re-generated. I think code could be more tiny and maintained more easily. Do you have any suggestion or think the original one is better? I'll start to update all patches once we have a better plan:-) Other points, 1. Macros is a good idea. It could make code more clear. 2. I am not fans of 'others', too. It's just a 'historical issue' from my early design. Thanks, Xu Wang > >> References to the other_node and other_prop structures could be made >> available for writing and reading data from them via methods. >> e.g. writing back the console (just a snippet from xmlgen.c) >> >> - console = xmlNewChild(root, NULL, BAD_CAST "console", >> NULL); >> - if (console == NULL) >> + other_node *console = NULL; >> + console = add_node_to_others(cdev->others->known_node, >> + "console", >> + NULL, /* text value of the >> xml element */ >> + "devices"); >> + >> + if (console == NULL) { >> + CU_DEBUG("Add tag failed."); >> return XML_ERROR; >> + } >> >> - xmlNewProp(console, BAD_CAST "type", >> - BAD_CAST >> - >> chardev_source_type_IDToStr(cdev->source_type)); >> + other_prop *cprop = NULL; >> + cprop = add_prop_to_others(console, >> + "type", >> + chardev_source_type_IDToStr( >> + cdev->source_type)); >> >> As you can see it is much closer to the known pluming as it used to be >> directly with xml and also hides the list internals (name & ids) used >> for referencing. >> >> I know that this would cause a rather large rework but I think that the >> usability would be much enhanced for everyone writing/fixing the >> provider code and overall would improve code stability in the long run. >> Please use this as an idea for improvement. > I think as part of the rewrite creating macros to replace commonly > cut-n-pasted code is a must. Currently there's numerous calls : > > + ddev->others = parse_data_to_others(ddev->others, > + dnode, > + 0, > + BAD_CAST "devices"); > + if (ddev->others == NULL) { > + CU_DEBUG("parse xml failed."); > + goto err; > + } > + > > or > > + if (seek_in_others(&ddev->others, > + -1, > + "source", > + TYPE_NODE, > + -1, > + (char *)dnode->name)) { > + ddev->source = fetch_from_others(&ddev->others, > + -1, > + "dir", > + TYPE_PROP, > + -1, > + "source"); > + > + > + if (ddev->source == NULL) { > + CU_DEBUG("no source dir"); > + goto err; > + } > + } > > There's macros that hide and do the error processing: > > GET_BASE_NODE(ddev, dnode, "devices", err); > > or > > ddev->source = GET_NODE_NODE(ddev, dnode, "source", err); > if (ddev->source) { > ddev->dir = GET_NODE_PROP(ddev->source, "dir", err); > ddev->startupPolicy = GET_NODE_PROP(ddev, "startupPolicy", err); > GET_NODE_NODE(ddev, ddev->source, "address", err); > if (ddev->address) { > } > } > > The various macros would handle the CU_DEBUG (and any other error) > processing. Similarly there would be PUT_BASE_NODE, PUT_NODE_NODE, > PUT_NODE_PROP. > > The macros aren't completely thought out, but I would hope you see their > value... > > >> Also note: The above is just email written code for reference and not >> thought to be bug free. :-) > Hah - bug free code... If it were all bug free we'd be out of a job :-) > > > I think we need to come up with an agreement on the architecture first > (currently patches 1-3 and then what was the original 21/48). > > Those should be submitted first without any other patches and without > necessarily being called except perhaps through xml_parse_test which > should be used to prove that the "new" parsing and generation does no > worse than the old (a 5th patch perhaps). > > Once those patches are in, then submit the patches that read/write > domain level data... Then submit the patches that read/write domain/mem > data... then domain/vcpu, etc. Also, rather than separate the xml > parsing & generating, keep it together for each type. > > Doing things in smaller bunches will be easier to review and less prone > to "other" changes causing issues. > > As an aside, I'm personally not a fan of the name 'others', but I don't > have a better suggestion :-) > > I do ask that you make the testing component of this a priority. Being > able to use xml_parse_test to validate is key. You may also want to add > code to it in order to print out elements that are "inactive" - e.g. all > the unknown stuff. Having visibility into what is not processed would be > a good thing. > > John > > >> Mit freundlichen Gr??en/Kind regards >> Boris Fiuczynski >> >> IBM Deutschland Research & Development GmbH >> Vorsitzender des Aufsichtsrats: Martina K?deritz >> Gesch?ftsf?hrung: Dirk Wittkopp >> Sitz der Gesellschaft: B?blingen >> Registergericht: Amtsgericht Stuttgart, HRB 243294 >> >> On 11/15/2013 01:23 AM, John Ferlan wrote: From fiuczy at linux.vnet.ibm.com Wed Nov 20 13:27:01 2013 From: fiuczy at linux.vnet.ibm.com (Boris Fiuczynski) Date: Wed, 20 Nov 2013 14:27:01 +0100 Subject: [Libvirt-cim] [PATCH 00/20] REWORK/PARIAL: Changes to solve unsupported tag issue In-Reply-To: <528BDCF9.8010109@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> <528A2B58.5010907@linux.vnet.ibm.com> <528BDCF9.8010109@redhat.com> Message-ID: <528CB8A5.6090702@linux.vnet.ibm.com> John and Xu, On 11/19/2013 10:49 PM, John Ferlan wrote: > On 11/18/2013 09:59 AM, Boris Fiuczynski wrote: >> John and Xu Wang, >> here are a few general observations from side: > > First off - I tend to find myself agreeing with Boris here. I think the > concept is important and necessary; however, I'm not convinced the > implementation goes far enough. > >> 1) I agree that it makes sense to preserve the unknown xml "entities" >> even so it can create complex scenarios and even new kinds of errors if >> unknown entities depend on known entities which get modified making them >> unusable for the unknown entities. This error would probably be the >> reversal of what is currently the problem when unknown entities simply >> disappear. > > Is there a more concrete example of "new kinds of errors if unknown > entities depend on known entities which get modified making them > unusable for the unknown entities" that can be given? Just for clarity. > I've read that line a few times and I'm still not sure :-) OK, let's take a look at device type disk. Since 1.0.2 the property sgio was added. Let's assume this is the unknown entity. sgio is only valid for property device "lun". If one now changes the property device to "disk" than the unknown entity sgio would cause an error when specified. > > >> 2) The implementation of putting hierarchical data (xml elements and >> properties) into a single "flat" list and storing the hierarchical >> dependencies with strings worries me. Wouldn't it make things easier to >> create the hierarchy in the others structure (it also would remove the >> need for the ids I guess!). > > The primary reason why I went through the trouble of adjusting the > commit was to learn a bit more about the code, but I also was trying to > figure out a way to split up the submit into more manageable chunks... > While going through the changes I found myself wondering what purpose > the parent values were serving and why we kept passing -1 for ID values. > > When I went to integrate the "device address" changes into this new > model I found myself beginning to think that perhaps the model used > there to grab the property key/value pairs from the node generically was > more suited for what you were trying to do. > > Since you're trying to protect against is someone changing/extending the > XML "node" to add more "properties" which are then not restored when > libvirt-cim goes to write out the XML, then why not take the next step > and just handle the whole hierarchy? > > >> The others structure is stored on the internal data object that >> represents a defined known xml entity. This defined known xml entity can >> itself be the parent for unknown properties (xml properites) and/or the >> parent for unknown sub entities (xml elements) where all these entities >> at the end of the "data parsing" would be flagged ACTIVE and the unknown >> ones INACTIVE. >> >> /* This could actually be just other_node */ >> struct others { >> struct other_node *known_node; /* this is the root representing >> the defined known xml element */ >> }; >> >> struct other_node { >> char *name; >> char *value; >> struct other_prop *properties; /* list of the nodes properties */ >> struct other_node *children; /* list with the direct sets of >> children data */ >> enum { >> ACTIVE, >> INACTIVE >> } status; >> }; >> >> struct other_prop { >> char *name; >> char *value; >> struct other_prop *properties; >> enum { >> ACTIVE, >> INACTIVE >> } status; >> }; >> >> Probably the above data structures could be streamlined even more when >> one starts to use these structures writing the code. The structures are >> very close to the xml and recursive methods could be used for writing >> the xml out. > > See the "device_address" structure and subsequent processing by > parse_device_address() and add_device_address_property() for more opaque > view on how to process the xml. > > If you take the concept and apply it to say "" - I think you'll > be able to create one structure for dominfo that contains one instance > of the read XML. Then using the active/inactive tagging you should be > able to tell what's been read/used by the libvirt-cim code. I'd even go > as far to say why not add a "CHANGED" status, but I'm not as clear on > the lifecycle with respect to how the code the code that ends up writing > things out works. My one concern with a single instance would be what > happens if it changes without libvirt-cim's knowledge? Not sure its > possible or supported. The idea to use one tree is fine with me. It would actually mean that the XML data structure would end up in a XML alike libvirt-cim data structure. This structure would be easier to convert back into the required xml document. If I understand you, John, correctly you also like to maintain the existing internal libvirt-cim data structures as they are today. If we don't maintain these data structures this would cause code changes at all rasd read and write methods. Not nice and more headaches. In addition I would like to suggest that for easier access into the new tree structure a reference to the corresponding tree node element should be stored as "other" (maybe rename it to node_ref) in the existing libvirt-cim data structure. A question comes to my mind about the usage of one tree: Does it create a problem for the helper methods (seek..., fetch..., ...) on instances that are not leafs (e.g. domain). I guess it would now be needed to know the hierarchy depth at which these methods stop. That is something Xu is the best to answer. > > I think the way the code is written now there's perhaps more than one > traversal of the tree, although without seeing in "in action" I cannot > be sure. > >> >> References to the other_node and other_prop structures could be made >> available for writing and reading data from them via methods. >> e.g. writing back the console (just a snippet from xmlgen.c) >> >> - console = xmlNewChild(root, NULL, BAD_CAST "console", >> NULL); >> - if (console == NULL) >> + other_node *console = NULL; >> + console = add_node_to_others(cdev->others->known_node, >> + "console", >> + NULL, /* text value of the >> xml element */ >> + "devices"); >> + >> + if (console == NULL) { >> + CU_DEBUG("Add tag failed."); >> return XML_ERROR; >> + } >> >> - xmlNewProp(console, BAD_CAST "type", >> - BAD_CAST >> - >> chardev_source_type_IDToStr(cdev->source_type)); >> + other_prop *cprop = NULL; >> + cprop = add_prop_to_others(console, >> + "type", >> + chardev_source_type_IDToStr( >> + cdev->source_type)); >> >> As you can see it is much closer to the known pluming as it used to be >> directly with xml and also hides the list internals (name & ids) used >> for referencing. >> >> I know that this would cause a rather large rework but I think that the >> usability would be much enhanced for everyone writing/fixing the >> provider code and overall would improve code stability in the long run. >> Please use this as an idea for improvement. > > I think as part of the rewrite creating macros to replace commonly > cut-n-pasted code is a must. Currently there's numerous calls : > > + ddev->others = parse_data_to_others(ddev->others, > + dnode, > + 0, > + BAD_CAST "devices"); > + if (ddev->others == NULL) { > + CU_DEBUG("parse xml failed."); > + goto err; > + } > + > > or > > + if (seek_in_others(&ddev->others, > + -1, > + "source", > + TYPE_NODE, > + -1, > + (char *)dnode->name)) { > + ddev->source = fetch_from_others(&ddev->others, > + -1, > + "dir", > + TYPE_PROP, > + -1, > + "source"); > + > + > + if (ddev->source == NULL) { > + CU_DEBUG("no source dir"); > + goto err; > + } > + } > > There's macros that hide and do the error processing: > > GET_BASE_NODE(ddev, dnode, "devices", err); > > or > > ddev->source = GET_NODE_NODE(ddev, dnode, "source", err); > if (ddev->source) { > ddev->dir = GET_NODE_PROP(ddev->source, "dir", err); > ddev->startupPolicy = GET_NODE_PROP(ddev, "startupPolicy", err); > GET_NODE_NODE(ddev, ddev->source, "address", err); > if (ddev->address) { > } > } > > The various macros would handle the CU_DEBUG (and any other error) > processing. Similarly there would be PUT_BASE_NODE, PUT_NODE_NODE, > PUT_NODE_PROP. > > The macros aren't completely thought out, but I would hope you see their > value... I think it is a good idea but suggest to wait and see how complex the helper methods end up to be. Instead of a macro it would maybe also make more sense regarding performance to write a single method e.g. for seek&fetch. > > >> Also note: The above is just email written code for reference and not >> thought to be bug free. :-) > > Hah - bug free code... If it were all bug free we'd be out of a job :-) > > > I think we need to come up with an agreement on the architecture first > (currently patches 1-3 and then what was the original 21/48). > > Those should be submitted first without any other patches and without > necessarily being called except perhaps through xml_parse_test which > should be used to prove that the "new" parsing and generation does no > worse than the old (a 5th patch perhaps). Good point! The xml_parse_test is a good validation tool for the first part. > > Once those patches are in, then submit the patches that read/write > domain level data... Then submit the patches that read/write domain/mem > data... then domain/vcpu, etc. Also, rather than separate the xml > parsing & generating, keep it together for each type. This seems like a valid and doable approach. Do you agree as well, Xu? > > Doing things in smaller bunches will be easier to review and less prone > to "other" changes causing issues. > > As an aside, I'm personally not a fan of the name 'others', but I don't > have a better suggestion :-) > > I do ask that you make the testing component of this a priority. Being > able to use xml_parse_test to validate is key. You may also want to add > code to it in order to print out elements that are "inactive" - e.g. all > the unknown stuff. Having visibility into what is not processed would be > a good thing. A smooth transition is key. Testing and comparison of old and new results seems crucial. > > John > > -- Mit freundlichen Gr??en/Kind regards Boris Fiuczynski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martina K?deritz Gesch?ftsf?hrung: Dirk Wittkopp Sitz der Gesellschaft: B?blingen Registergericht: Amtsgericht Stuttgart, HRB 243294 From mihajlov at linux.vnet.ibm.com Wed Nov 20 18:04:24 2013 From: mihajlov at linux.vnet.ibm.com (Viktor Mihajlovski) Date: Wed, 20 Nov 2013 19:04:24 +0100 Subject: [Libvirt-cim] [PATCH 2/5] libxkutil: Support for device addresses In-Reply-To: <528BD1FE.6070306@redhat.com> References: <1381764584-18025-1-git-send-email-mihajlov@linux.vnet.ibm.com> <1381764584-18025-3-git-send-email-mihajlov@linux.vnet.ibm.com> <528BD1FE.6070306@redhat.com> Message-ID: <528CF9A8.80101@linux.vnet.ibm.com> On 11/19/2013 10:02 PM, John Ferlan wrote: > On 10/14/2013 11:29 AM, Viktor Mihajlovski wrote: >> New data type for device addresses based on the generic >> "address" element of the libvirt domain XML. >> Contains XML parsing and generation code for device addresses. >> >> Signed-off-by: Viktor Mihajlovski >> --- >> libxkutil/device_parsing.c | 113 ++++++++++++++++++++++++++++++++++++++++++++ >> libxkutil/device_parsing.h | 11 +++++ >> libxkutil/xmlgen.c | 28 +++++++++++ >> 3 files changed, 152 insertions(+) >> > > Although this is already pushed, while I was working through Xu's patch > set I think I found a missed case. > > The parsing code handles "parse_fs_device", "parse_block_device", and > "parse_net_device"; however, the generation code does "disk_block_xml", > "disk_file_xml", and "net_xml" missing out on "disk_fs_xml". true ... will send a patch fixing it. > > The "parse_block_device" handles both "source=file" and "source=dev", > while "disk_xml" checks for DISK_PHY or DISK_FILE before parceling out > to the corresponding subroutine. > > John > -- Mit freundlichen Gr??en/Kind Regards Viktor Mihajlovski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martina K?deritz Gesch?ftsf?hrung: Dirk Wittkopp Sitz der Gesellschaft: B?blingen Registergericht: Amtsgericht Stuttgart, HRB 243294 From mihajlov at linux.vnet.ibm.com Wed Nov 20 18:09:41 2013 From: mihajlov at linux.vnet.ibm.com (Viktor Mihajlovski) Date: Wed, 20 Nov 2013 19:09:41 +0100 Subject: [Libvirt-cim] [PATCH 00/20] REWORK/PARIAL: Changes to solve unsupported tag issue In-Reply-To: <52866ACE.8050803@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> <528645FA.7020305@linux.vnet.ibm.com> <52866ACE.8050803@redhat.com> Message-ID: <528CFAE5.8000304@linux.vnet.ibm.com> On 11/15/2013 07:41 PM, John Ferlan wrote: >> However I need to report an issue running on s390, cimprovagt >> core dumps, so I need to investigate further and ask to >> please hold off until I figure out the reason. Thanks! >> > > I found investigating cimprovagt to be very painful, hence the reason > why I redid 1-15 a bit. I'd suggest trying to apply 1-3 first - make > sure they work. Then 4-13 to make sure they work. Then go slower on > 14-20. I found that 14/15 were the most problematic... 16-18 were > mechanical. 19 seemed to be harmless; however, who knows. well, it turns out that I was hitting a low memory condition, maybe a side effect of patches, redoing the tests with more memory worked well, which it should anyway because no "other" libvirt XML is written back so far. So, no worries, especially as the patches are under discussion anyway. -- Mit freundlichen Gr??en/Kind Regards Viktor Mihajlovski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martina K?deritz Gesch?ftsf?hrung: Dirk Wittkopp Sitz der Gesellschaft: B?blingen Registergericht: Amtsgericht Stuttgart, HRB 243294 From mihajlov at linux.vnet.ibm.com Wed Nov 20 18:43:17 2013 From: mihajlov at linux.vnet.ibm.com (Viktor Mihajlovski) Date: Wed, 20 Nov 2013 19:43:17 +0100 Subject: [Libvirt-cim] [PATCH] libxkutil: Added missing address element for filesystem 'disk' Message-ID: <1384972997-24300-1-git-send-email-mihajlov@linux.vnet.ibm.com> This was an omission from the original patch series adding device address support. For disks of type filesystem the addresses were parsed however not written back. Signed-off-by: Viktor Mihajlovski --- libxkutil/xmlgen.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/libxkutil/xmlgen.c b/libxkutil/xmlgen.c index 40e2905..931f0c9 100644 --- a/libxkutil/xmlgen.c +++ b/libxkutil/xmlgen.c @@ -342,6 +342,9 @@ static const char *disk_fs_xml(xmlNodePtr root, struct disk_device *dev) return XML_ERROR; xmlNewProp(tmp, BAD_CAST "dir", BAD_CAST dev->virtual_dev); + if (dev->address.ct > 0) + return device_address_xml(fs, &dev->address); + return NULL; } -- 1.7.9.5 From mihajlov at linux.vnet.ibm.com Wed Nov 20 19:19:09 2013 From: mihajlov at linux.vnet.ibm.com (Viktor Mihajlovski) Date: Wed, 20 Nov 2013 20:19:09 +0100 Subject: [Libvirt-cim] [PATCH] live.full_hostname: Adjust mechanism to get FQDN In-Reply-To: <1384867543-8939-1-git-send-email-jferlan@redhat.com> References: <1384867543-8939-1-git-send-email-jferlan@redhat.com> Message-ID: <528D0B2D.5060702@linux.vnet.ibm.com> On 11/19/2013 02:25 PM, John Ferlan wrote: > Using just socket.gethostbyaddr(socket.gethostname())[0] caused some > issues recently in one of my DHCP testing environments. Breaking down > the calls showed the following: > >>>> socket.gethostname() > 'dhcp-186-211.bos.redhat.com' >>>> socket.gethostbyname("dhcp-186-211.bos.redhat.com") > '10.16.186.211' >>>> socket.gethostbyaddr(socket.gethostname())[0] > Traceback (most recent call last): > File "", line 1, in > socket.herror: [Errno 1] Unknown host > > While just a socket.gethostname() could have worked, using the > socket.getfqdn() seemed to be safer just in case. > --- > lib/VirtLib/live.py | 7 +++++-- > 1 file changed, 5 insertions(+), 2 deletions(-) > > diff --git a/lib/VirtLib/live.py b/lib/VirtLib/live.py > index c929e71..6a10474 100644 > --- a/lib/VirtLib/live.py > +++ b/lib/VirtLib/live.py > @@ -100,6 +100,9 @@ def hostname(server): > return out > > def full_hostname(server): > - """To return the fully qualifiec domain name(FQDN) of the system""" > + """To return the fully qualified domain name(FQDN) of the system""" > > - return socket.gethostbyaddr(socket.gethostname())[0] > + if socket.getfqdn().find('.') >= 0: > + return socket.getfqdn() > + else: > + return socket.gethostbyaddr(socket.gethostname())[0] > The server parameter seems to be entirely superfluous, which makes me believe that the original intention might have been something like (roughly) return socket.gethostbyaddr(server) which by itself will probably not solve the reverse lookup issue you seem to have experienced -- Mit freundlichen Gr??en/Kind Regards Viktor Mihajlovski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martina K?deritz Gesch?ftsf?hrung: Dirk Wittkopp Sitz der Gesellschaft: B?blingen Registergericht: Amtsgericht Stuttgart, HRB 243294 From jferlan at redhat.com Wed Nov 20 19:49:51 2013 From: jferlan at redhat.com (John Ferlan) Date: Wed, 20 Nov 2013 14:49:51 -0500 Subject: [Libvirt-cim] [PATCH 00/20] REWORK/PARIAL: Changes to solve unsupported tag issue In-Reply-To: <528CFAE5.8000304@linux.vnet.ibm.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> <528645FA.7020305@linux.vnet.ibm.com> <52866ACE.8050803@redhat.com> <528CFAE5.8000304@linux.vnet.ibm.com> Message-ID: <528D125F.1080502@redhat.com> On 11/20/2013 01:09 PM, Viktor Mihajlovski wrote: > On 11/15/2013 07:41 PM, John Ferlan wrote: > >>> However I need to report an issue running on s390, cimprovagt >>> core dumps, so I need to investigate further and ask to >>> please hold off until I figure out the reason. Thanks! >>> >> >> I found investigating cimprovagt to be very painful, hence the reason >> why I redid 1-15 a bit. I'd suggest trying to apply 1-3 first - make >> sure they work. Then 4-13 to make sure they work. Then go slower on >> 14-20. I found that 14/15 were the most problematic... 16-18 were >> mechanical. 19 seemed to be harmless; however, who knows. > > well, it turns out that I was hitting a low memory condition, > maybe a side effect of patches, redoing the tests with more memory > worked well, which it should anyway because no "other" libvirt XML is > written back so far. So, no worries, especially as the patches are > under discussion anyway. > That brings up an interesting test - how much memory is used by the current algorithm and how much gets used by the new algorithm... Not that there's a leak, I just think more memory is held onto than necessary. If each device has its own parse_data tree, then there's a lot of unnecessary duplication and even worse unused fields. Unfortunately when the data is then fetched - we don't add another pile of memory which isn't freed. At least this new code does a better job at failing on memory allocation compared to many other paths in the code... John From jferlan at redhat.com Wed Nov 20 20:24:24 2013 From: jferlan at redhat.com (John Ferlan) Date: Wed, 20 Nov 2013 15:24:24 -0500 Subject: [Libvirt-cim] [PATCH 00/20] REWORK/PARIAL: Changes to solve unsupported tag issue In-Reply-To: <528C74A8.9010205@gmail.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> <528A2B58.5010907@linux.vnet.ibm.com> <528BDCF9.8010109@redhat.com> <528C74A8.9010205@gmail.com> Message-ID: <528D1A78.6090703@redhat.com> On 11/20/2013 03:36 AM, Xu Wang wrote: > Dear John and Boris, > I am considering pros/cons of suggestion from Boris. If the parsing > process likes this, > 1. Traversal the whole xml and save all data into data structure Boris > describe above; I hope the issue Viktor ran into with running out of memory shows that a duplicitous design is problematic. If you have to - rather than strdup() value, set INACTIVE, and return (unchecked) strdup()'d memory - it would have been just as easy to return the memory already allocated, set tmp->value = NULL, and set INACTIVE. That way you know it's "managed" by something else. The free(NULL) is a no-op anyway... Be sure to use/change the "xml_parse_test" in order to validate. In fact that print function you created could be called by xml_parse_test - at least temporarily. At first everything would be unknown. As you update parse functions, you should be able to prove that you've gone from unknown to known... Eventually your known should be the same output as what one could get 'today' if they ran the test (I forget the exact switches, but you'll get to know them). > 2. Calling ***_parse() functions in device_parsing.c fetch data from the > data structure > generated in step 1. I think 2 & 4/5 need to be intermingled. That is when you update 'parse_fs_device', also update 'disk_fs_xml'. I'd change 'parse_block_device', 'disk_block_xml', and 'disk_file_xml' all at once. Change all the network functions together. Etc. > 3. All kinds of resource operations. > (Here I want to enhance the power of libvirt-cim after these patches > merged. Some > new name and value pairs could be passed from API and Management could > be done. > The work next step maybe more complex and I started to think about it) I'm not exactly sure what this means, but it seems like this would be after step 5. Although it cannot be ignored or forgotten when devising the mechanism to parse/generate XML The change validity concern when there's unknown > 4. Restore xml after operation, ***_xml() restore all data in the > virt_devices fetched from > data structure > 5. Based on data structure restored, new xml re-generated. What's the difference between 4 & 5? > > I think code could be more tiny and maintained more easily. > Do you have any suggestion or think the original one is better? I'll > start to update all > patches once we have a better plan:-) Be conservative - I know it's hard to not make all the changes at once, but let's get the infrastructure right first. Updating the various _parse() and _xml() modules is a rote operation, but splatting out 50 patches at a time is tough to review/test... John > > Other points, > 1. Macros is a good idea. It could make code more clear. > 2. I am not fans of 'others', too. It's just a 'historical issue' from > my early design. It's the "unmanaged" or "unrealized" list... John > > Thanks, > Xu Wang From jferlan at redhat.com Wed Nov 20 21:10:15 2013 From: jferlan at redhat.com (John Ferlan) Date: Wed, 20 Nov 2013 16:10:15 -0500 Subject: [Libvirt-cim] [PATCH 00/20] REWORK/PARIAL: Changes to solve unsupported tag issue In-Reply-To: <528CB8A5.6090702@linux.vnet.ibm.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> <528A2B58.5010907@linux.vnet.ibm.com> <528BDCF9.8010109@redhat.com> <528CB8A5.6090702@linux.vnet.ibm.com> Message-ID: <528D2537.3010408@redhat.com> On 11/20/2013 08:27 AM, Boris Fiuczynski wrote: > John and Xu, > > On 11/19/2013 10:49 PM, John Ferlan wrote: >> On 11/18/2013 09:59 AM, Boris Fiuczynski wrote: >>> John and Xu Wang, >>> here are a few general observations from side: >> >> First off - I tend to find myself agreeing with Boris here. I think the >> concept is important and necessary; however, I'm not convinced the >> implementation goes far enough. >> >>> 1) I agree that it makes sense to preserve the unknown xml "entities" >>> even so it can create complex scenarios and even new kinds of errors if >>> unknown entities depend on known entities which get modified making them >>> unusable for the unknown entities. This error would probably be the >>> reversal of what is currently the problem when unknown entities simply >>> disappear. >> >> Is there a more concrete example of "new kinds of errors if unknown >> entities depend on known entities which get modified making them >> unusable for the unknown entities" that can be given? Just for clarity. >> I've read that line a few times and I'm still not sure :-) > OK, let's take a look at device type disk. > Since 1.0.2 the property sgio was added. Let's assume this is the > unknown entity. sgio is only valid for property device "lun". > If one now changes the property device to "disk" than the unknown entity > sgio would cause an error when specified. > Ah - I see. Not only do you have to manage the properties you have to know how to use them as well and all their rules. I forgot about that. I came from HP/HPVM and yes, this brings back all sorts of memories... Seems like in this case, when/if the property was changed from "lun" to "disk" - code would have to search that 'others' list for the "sgio" property and know how to handle adjusting it. That'll get tricky... >> >> >>> 2) The implementation of putting hierarchical data (xml elements and >>> properties) into a single "flat" list and storing the hierarchical >>> dependencies with strings worries me. Wouldn't it make things easier to >>> create the hierarchy in the others structure (it also would remove the >>> need for the ids I guess!). >> >> The primary reason why I went through the trouble of adjusting the >> commit was to learn a bit more about the code, but I also was trying to >> figure out a way to split up the submit into more manageable chunks... >> While going through the changes I found myself wondering what purpose >> the parent values were serving and why we kept passing -1 for ID values. >> >> When I went to integrate the "device address" changes into this new >> model I found myself beginning to think that perhaps the model used >> there to grab the property key/value pairs from the node generically was >> more suited for what you were trying to do. >> >> Since you're trying to protect against is someone changing/extending the >> XML "node" to add more "properties" which are then not restored when >> libvirt-cim goes to write out the XML, then why not take the next step >> and just handle the whole hierarchy? >> >> >>> The others structure is stored on the internal data object that >>> represents a defined known xml entity. This defined known xml entity can >>> itself be the parent for unknown properties (xml properites) and/or the >>> parent for unknown sub entities (xml elements) where all these entities >>> at the end of the "data parsing" would be flagged ACTIVE and the unknown >>> ones INACTIVE. >>> >>> /* This could actually be just other_node */ >>> struct others { >>> struct other_node *known_node; /* this is the root representing >>> the defined known xml element */ >>> }; >>> >>> struct other_node { >>> char *name; >>> char *value; >>> struct other_prop *properties; /* list of the nodes >>> properties */ >>> struct other_node *children; /* list with the direct sets of >>> children data */ >>> enum { >>> ACTIVE, >>> INACTIVE >>> } status; >>> }; >>> >>> struct other_prop { >>> char *name; >>> char *value; >>> struct other_prop *properties; >>> enum { >>> ACTIVE, >>> INACTIVE >>> } status; >>> }; >>> >>> Probably the above data structures could be streamlined even more when >>> one starts to use these structures writing the code. The structures are >>> very close to the xml and recursive methods could be used for writing >>> the xml out. >> >> See the "device_address" structure and subsequent processing by >> parse_device_address() and add_device_address_property() for more opaque >> view on how to process the xml. >> >> If you take the concept and apply it to say "" - I think you'll >> be able to create one structure for dominfo that contains one instance >> of the read XML. Then using the active/inactive tagging you should be >> able to tell what's been read/used by the libvirt-cim code. I'd even go >> as far to say why not add a "CHANGED" status, but I'm not as clear on >> the lifecycle with respect to how the code the code that ends up writing >> things out works. My one concern with a single instance would be what >> happens if it changes without libvirt-cim's knowledge? Not sure its >> possible or supported. > The idea to use one tree is fine with me. It would actually mean that > the XML data structure would end up in a XML alike libvirt-cim data > structure. This structure would be easier to convert back into the > required xml document. If I understand you, John, correctly you also > like to maintain the existing internal libvirt-cim data structures as > they are today. If we don't maintain these data structures this would > cause code changes at all rasd read and write methods. Not nice and more > headaches. I hadn't given much thought to adjusting the internal structures at all. I agree keeping the change/churn to a minimum is a goal. As alluded to in my other response as we fill these data structures and grab memory/data from the other or unmanaged list - we probably shouldn't strdup() the value. Instead just take it, clear the value field, and mark the entry as managed. I haven't fully thought out the ramifications yet though... When we go to update/generate the output the opposite happens - we know the value moved from "others" to some "*_device" field - now we have to grab whatever is current, do some sort of validation... Thus our status field is UNMANAGED, MANAGED, CHANGED... When we come across CHANGED we may have to do a couple of handstands based on whether other properties rely on this particular property changing (as from above). Either that or when it changes we find/manage the property on the others list and deal with it... > > In addition I would like to suggest that for easier access into the new > tree structure a reference to the corresponding tree node element should > be stored as "other" (maybe rename it to node_ref) in the existing > libvirt-cim data structure. That works too (rather than borrowing and returning). > A question comes to my mind about the usage of one tree: Does it create > a problem for the helper methods (seek..., fetch..., ...) on instances > that are not leafs (e.g. domain). I guess it would now be needed to know > the hierarchy depth at which these methods stop. That is something Xu is > the best to answer. > Right! and a bit of prototyping... Validate the algorithm/thoughts work with 'domain' and perhaps 'file'/'block'... >> >> I think the way the code is written now there's perhaps more than one >> traversal of the tree, although without seeing in "in action" I cannot >> be sure. >> >>> >>> References to the other_node and other_prop structures could be made >>> available for writing and reading data from them via methods. >>> e.g. writing back the console (just a snippet from xmlgen.c) >>> >>> - console = xmlNewChild(root, NULL, BAD_CAST "console", >>> NULL); >>> - if (console == NULL) >>> + other_node *console = NULL; >>> + console = add_node_to_others(cdev->others->known_node, >>> + "console", >>> + NULL, /* text value of the >>> xml element */ >>> + "devices"); >>> + >>> + if (console == NULL) { >>> + CU_DEBUG("Add tag failed."); >>> return XML_ERROR; >>> + } >>> >>> - xmlNewProp(console, BAD_CAST "type", >>> - BAD_CAST >>> - >>> chardev_source_type_IDToStr(cdev->source_type)); >>> + other_prop *cprop = NULL; >>> + cprop = add_prop_to_others(console, >>> + "type", >>> + chardev_source_type_IDToStr( >>> + cdev->source_type)); >>> >>> As you can see it is much closer to the known pluming as it used to be >>> directly with xml and also hides the list internals (name & ids) used >>> for referencing. >>> >>> I know that this would cause a rather large rework but I think that the >>> usability would be much enhanced for everyone writing/fixing the >>> provider code and overall would improve code stability in the long run. >>> Please use this as an idea for improvement. >> >> I think as part of the rewrite creating macros to replace commonly >> cut-n-pasted code is a must. Currently there's numerous calls : >> >> + ddev->others = parse_data_to_others(ddev->others, >> + dnode, >> + 0, >> + BAD_CAST "devices"); >> + if (ddev->others == NULL) { >> + CU_DEBUG("parse xml failed."); >> + goto err; >> + } >> + >> >> or >> >> + if (seek_in_others(&ddev->others, >> + -1, >> + "source", >> + TYPE_NODE, >> + -1, >> + (char *)dnode->name)) { >> + ddev->source = fetch_from_others(&ddev->others, >> + -1, >> + "dir", >> + TYPE_PROP, >> + -1, >> + "source"); >> + >> + >> + if (ddev->source == NULL) { >> + CU_DEBUG("no source dir"); >> + goto err; >> + } >> + } >> >> There's macros that hide and do the error processing: >> >> GET_BASE_NODE(ddev, dnode, "devices", err); >> >> or >> >> ddev->source = GET_NODE_NODE(ddev, dnode, "source", err); >> if (ddev->source) { >> ddev->dir = GET_NODE_PROP(ddev->source, "dir", err); >> ddev->startupPolicy = GET_NODE_PROP(ddev, "startupPolicy", err); >> GET_NODE_NODE(ddev, ddev->source, "address", err); >> if (ddev->address) { >> } >> } >> >> The various macros would handle the CU_DEBUG (and any other error) >> processing. Similarly there would be PUT_BASE_NODE, PUT_NODE_NODE, >> PUT_NODE_PROP. >> >> The macros aren't completely thought out, but I would hope you see their >> value... > I think it is a good idea but suggest to wait and see how complex the > helper methods end up to be. Instead of a macro it would maybe also make > more sense regarding performance to write a single method e.g. for > seek&fetch. Anything that reduces cut-n-paste of the same 10-20 lines is better. I'm a big code reuse proponent. I chose macros because it's easier to have them jump to failure points rather than checking routine status. John >> >> >>> Also note: The above is just email written code for reference and not >>> thought to be bug free. :-) >> >> Hah - bug free code... If it were all bug free we'd be out of a job :-) >> >> >> I think we need to come up with an agreement on the architecture first >> (currently patches 1-3 and then what was the original 21/48). >> >> Those should be submitted first without any other patches and without >> necessarily being called except perhaps through xml_parse_test which >> should be used to prove that the "new" parsing and generation does no >> worse than the old (a 5th patch perhaps). > Good point! The xml_parse_test is a good validation tool for the first > part. >> >> Once those patches are in, then submit the patches that read/write >> domain level data... Then submit the patches that read/write domain/mem >> data... then domain/vcpu, etc. Also, rather than separate the xml >> parsing & generating, keep it together for each type. > This seems like a valid and doable approach. Do you agree as well, Xu? >> >> Doing things in smaller bunches will be easier to review and less prone >> to "other" changes causing issues. >> >> As an aside, I'm personally not a fan of the name 'others', but I don't >> have a better suggestion :-) >> >> I do ask that you make the testing component of this a priority. Being >> able to use xml_parse_test to validate is key. You may also want to add >> code to it in order to print out elements that are "inactive" - e.g. all >> the unknown stuff. Having visibility into what is not processed would be >> a good thing. > A smooth transition is key. Testing and comparison of old and new > results seems crucial. >> >> John >> >> > > > From gesaint at linux.vnet.ibm.com Thu Nov 21 03:15:45 2013 From: gesaint at linux.vnet.ibm.com (Xu Wang) Date: Thu, 21 Nov 2013 11:15:45 +0800 Subject: [Libvirt-cim] [PATCH 00/20] REWORK/PARIAL: Changes to solve unsupported tag issue In-Reply-To: <528D2537.3010408@redhat.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> <528A2B58.5010907@linux.vnet.ibm.com> <528BDCF9.8010109@redhat.com> <528CB8A5.6090702@linux.vnet.ibm.com> <528D2537.3010408@redhat.com> Message-ID: <528D7AE1.3000202@linux.vnet.ibm.com> ? 2013/11/21 5:10, John Ferlan ??: > On 11/20/2013 08:27 AM, Boris Fiuczynski wrote: >> John and Xu, >> >> On 11/19/2013 10:49 PM, John Ferlan wrote: >>> On 11/18/2013 09:59 AM, Boris Fiuczynski wrote: >>>> John and Xu Wang, >>>> here are a few general observations from side: >>> First off - I tend to find myself agreeing with Boris here. I think the >>> concept is important and necessary; however, I'm not convinced the >>> implementation goes far enough. >>> >>>> 1) I agree that it makes sense to preserve the unknown xml "entities" >>>> even so it can create complex scenarios and even new kinds of errors if >>>> unknown entities depend on known entities which get modified making them >>>> unusable for the unknown entities. This error would probably be the >>>> reversal of what is currently the problem when unknown entities simply >>>> disappear. >>> Is there a more concrete example of "new kinds of errors if unknown >>> entities depend on known entities which get modified making them >>> unusable for the unknown entities" that can be given? Just for clarity. >>> I've read that line a few times and I'm still not sure :-) >> OK, let's take a look at device type disk. >> Since 1.0.2 the property sgio was added. Let's assume this is the >> unknown entity. sgio is only valid for property device "lun". >> If one now changes the property device to "disk" than the unknown entity >> sgio would cause an error when specified. >> > Ah - I see. Not only do you have to manage the properties you have to > know how to use them as well and all their rules. I forgot about that. I > came from HP/HPVM and yes, this brings back all sorts of memories... > > Seems like in this case, when/if the property was changed from "lun" to > "disk" - code would have to search that 'others' list for the "sgio" > property and know how to handle adjusting it. That'll get tricky... In fact there is lots of hacking code like this in libvirt-cim. My suggestion is, if some node was changed (such as "lun" to "disk"), its unknown sub-entities should be marked as INACTIVE to avoid that issue. That is, we couldn't keep the validity of unknown entities, so maybe let libvirt-cim just do like now is a better choice. If you think sometimes the idea above still could miss something useful, some hack code need to be added to handle issue like this. >>> >>>> 2) The implementation of putting hierarchical data (xml elements and >>>> properties) into a single "flat" list and storing the hierarchical >>>> dependencies with strings worries me. Wouldn't it make things easier to >>>> create the hierarchy in the others structure (it also would remove the >>>> need for the ids I guess!). >>> The primary reason why I went through the trouble of adjusting the >>> commit was to learn a bit more about the code, but I also was trying to >>> figure out a way to split up the submit into more manageable chunks... >>> While going through the changes I found myself wondering what purpose >>> the parent values were serving and why we kept passing -1 for ID values. >>> >>> When I went to integrate the "device address" changes into this new >>> model I found myself beginning to think that perhaps the model used >>> there to grab the property key/value pairs from the node generically was >>> more suited for what you were trying to do. >>> >>> Since you're trying to protect against is someone changing/extending the >>> XML "node" to add more "properties" which are then not restored when >>> libvirt-cim goes to write out the XML, then why not take the next step >>> and just handle the whole hierarchy? >>> >>> >>>> The others structure is stored on the internal data object that >>>> represents a defined known xml entity. This defined known xml entity can >>>> itself be the parent for unknown properties (xml properites) and/or the >>>> parent for unknown sub entities (xml elements) where all these entities >>>> at the end of the "data parsing" would be flagged ACTIVE and the unknown >>>> ones INACTIVE. >>>> >>>> /* This could actually be just other_node */ >>>> struct others { >>>> struct other_node *known_node; /* this is the root representing >>>> the defined known xml element */ >>>> }; >>>> >>>> struct other_node { >>>> char *name; >>>> char *value; >>>> struct other_prop *properties; /* list of the nodes >>>> properties */ >>>> struct other_node *children; /* list with the direct sets of >>>> children data */ >>>> enum { >>>> ACTIVE, >>>> INACTIVE >>>> } status; >>>> }; >>>> >>>> struct other_prop { >>>> char *name; >>>> char *value; >>>> struct other_prop *properties; >>>> enum { >>>> ACTIVE, >>>> INACTIVE >>>> } status; >>>> }; >>>> >>>> Probably the above data structures could be streamlined even more when >>>> one starts to use these structures writing the code. The structures are >>>> very close to the xml and recursive methods could be used for writing >>>> the xml out. >>> See the "device_address" structure and subsequent processing by >>> parse_device_address() and add_device_address_property() for more opaque >>> view on how to process the xml. >>> >>> If you take the concept and apply it to say "" - I think you'll >>> be able to create one structure for dominfo that contains one instance >>> of the read XML. Then using the active/inactive tagging you should be >>> able to tell what's been read/used by the libvirt-cim code. I'd even go >>> as far to say why not add a "CHANGED" status, but I'm not as clear on >>> the lifecycle with respect to how the code the code that ends up writing >>> things out works. My one concern with a single instance would be what >>> happens if it changes without libvirt-cim's knowledge? Not sure its >>> possible or supported. >> The idea to use one tree is fine with me. It would actually mean that >> the XML data structure would end up in a XML alike libvirt-cim data >> structure. This structure would be easier to convert back into the >> required xml document. If I understand you, John, correctly you also >> like to maintain the existing internal libvirt-cim data structures as >> they are today. If we don't maintain these data structures this would >> cause code changes at all rasd read and write methods. Not nice and more >> headaches. > I hadn't given much thought to adjusting the internal structures at all. > I agree keeping the change/churn to a minimum is a goal. > > As alluded to in my other response as we fill these data structures and > grab memory/data from the other or unmanaged list - we probably > shouldn't strdup() the value. Instead just take it, clear the value > field, and mark the entry as managed. I haven't fully thought out the > ramifications yet though... > > When we go to update/generate the output the opposite happens - we know > the value moved from "others" to some "*_device" field - now we have to > grab whatever is current, do some sort of validation... > > Thus our status field is UNMANAGED, MANAGED, CHANGED... When we come > across CHANGED we may have to do a couple of handstands based on whether > other properties rely on this particular property changing (as from > above). Either that or when it changes we find/manage the property on > the others list and deal with it... > >> In addition I would like to suggest that for easier access into the new >> tree structure a reference to the corresponding tree node element should >> be stored as "other" (maybe rename it to node_ref) in the existing >> libvirt-cim data structure. > That works too (rather than borrowing and returning). That's a good idea. I think using 'strdup()' less could save memory space and running time. > >> A question comes to my mind about the usage of one tree: Does it create >> a problem for the helper methods (seek..., fetch..., ...) on instances >> that are not leafs (e.g. domain). I guess it would now be needed to know >> the hierarchy depth at which these methods stop. That is something Xu is >> the best to answer. >> > Right! and a bit of prototyping... Validate the algorithm/thoughts work > with 'domain' and perhaps 'file'/'block'... > >>> I think the way the code is written now there's perhaps more than one >>> traversal of the tree, although without seeing in "in action" I cannot >>> be sure. >>> >>>> References to the other_node and other_prop structures could be made >>>> available for writing and reading data from them via methods. >>>> e.g. writing back the console (just a snippet from xmlgen.c) >>>> >>>> - console = xmlNewChild(root, NULL, BAD_CAST "console", >>>> NULL); >>>> - if (console == NULL) >>>> + other_node *console = NULL; >>>> + console = add_node_to_others(cdev->others->known_node, >>>> + "console", >>>> + NULL, /* text value of the >>>> xml element */ >>>> + "devices"); >>>> + >>>> + if (console == NULL) { >>>> + CU_DEBUG("Add tag failed."); >>>> return XML_ERROR; >>>> + } >>>> >>>> - xmlNewProp(console, BAD_CAST "type", >>>> - BAD_CAST >>>> - >>>> chardev_source_type_IDToStr(cdev->source_type)); >>>> + other_prop *cprop = NULL; >>>> + cprop = add_prop_to_others(console, >>>> + "type", >>>> + chardev_source_type_IDToStr( >>>> + cdev->source_type)); >>>> >>>> As you can see it is much closer to the known pluming as it used to be >>>> directly with xml and also hides the list internals (name & ids) used >>>> for referencing. >>>> >>>> I know that this would cause a rather large rework but I think that the >>>> usability would be much enhanced for everyone writing/fixing the >>>> provider code and overall would improve code stability in the long run. >>>> Please use this as an idea for improvement. >>> I think as part of the rewrite creating macros to replace commonly >>> cut-n-pasted code is a must. Currently there's numerous calls : >>> >>> + ddev->others = parse_data_to_others(ddev->others, >>> + dnode, >>> + 0, >>> + BAD_CAST "devices"); >>> + if (ddev->others == NULL) { >>> + CU_DEBUG("parse xml failed."); >>> + goto err; >>> + } >>> + >>> >>> or >>> >>> + if (seek_in_others(&ddev->others, >>> + -1, >>> + "source", >>> + TYPE_NODE, >>> + -1, >>> + (char *)dnode->name)) { >>> + ddev->source = fetch_from_others(&ddev->others, >>> + -1, >>> + "dir", >>> + TYPE_PROP, >>> + -1, >>> + "source"); >>> + >>> + >>> + if (ddev->source == NULL) { >>> + CU_DEBUG("no source dir"); >>> + goto err; >>> + } >>> + } >>> >>> There's macros that hide and do the error processing: >>> >>> GET_BASE_NODE(ddev, dnode, "devices", err); >>> >>> or >>> >>> ddev->source = GET_NODE_NODE(ddev, dnode, "source", err); >>> if (ddev->source) { >>> ddev->dir = GET_NODE_PROP(ddev->source, "dir", err); >>> ddev->startupPolicy = GET_NODE_PROP(ddev, "startupPolicy", err); >>> GET_NODE_NODE(ddev, ddev->source, "address", err); >>> if (ddev->address) { >>> } >>> } >>> >>> The various macros would handle the CU_DEBUG (and any other error) >>> processing. Similarly there would be PUT_BASE_NODE, PUT_NODE_NODE, >>> PUT_NODE_PROP. >>> >>> The macros aren't completely thought out, but I would hope you see their >>> value... >> I think it is a good idea but suggest to wait and see how complex the >> helper methods end up to be. Instead of a macro it would maybe also make >> more sense regarding performance to write a single method e.g. for >> seek&fetch. > Anything that reduces cut-n-paste of the same 10-20 lines is better. > I'm a big code reuse proponent. I chose macros because it's easier to > have them jump to failure points rather than checking routine status. > > John >>> >>>> Also note: The above is just email written code for reference and not >>>> thought to be bug free. :-) >>> Hah - bug free code... If it were all bug free we'd be out of a job :-) >>> >>> >>> I think we need to come up with an agreement on the architecture first >>> (currently patches 1-3 and then what was the original 21/48). >>> >>> Those should be submitted first without any other patches and without >>> necessarily being called except perhaps through xml_parse_test which >>> should be used to prove that the "new" parsing and generation does no >>> worse than the old (a 5th patch perhaps). >> Good point! The xml_parse_test is a good validation tool for the first >> part. >>> Once those patches are in, then submit the patches that read/write >>> domain level data... Then submit the patches that read/write domain/mem >>> data... then domain/vcpu, etc. Also, rather than separate the xml >>> parsing & generating, keep it together for each type. >> This seems like a valid and doable approach. Do you agree as well, Xu? I can't agree with you more:-) >>> Doing things in smaller bunches will be easier to review and less prone >>> to "other" changes causing issues. >>> >>> As an aside, I'm personally not a fan of the name 'others', but I don't >>> have a better suggestion :-) >>> >>> I do ask that you make the testing component of this a priority. Being >>> able to use xml_parse_test to validate is key. You may also want to add >>> code to it in order to print out elements that are "inactive" - e.g. all >>> the unknown stuff. Having visibility into what is not processed would be >>> a good thing. >> A smooth transition is key. Testing and comparison of old and new >> results seems crucial. >>> John >>> >>> >> >> And the whole process like this? 1. Traverse content of xml -> build references -> save into 'others' (just name it temporarily) 2. Fetch/handle data into virt_device structure 3. Resource operations (just like original, some change may need to be done such as status mark) 4. Restore the data in the virt_devices back to the xml 5. Regenerate the xml based on the references (status mainly) 6. The following steps (just like original) Some maros (functions) like this maybe needed, BUILD_REFERENCES, GET_NODE, GET_PROP, UPDATE_STATUS, ...... Thanks, Xu Wang > _______________________________________________ > Libvirt-cim mailing list > Libvirt-cim at redhat.com > https://www.redhat.com/mailman/listinfo/libvirt-cim > From jferlan at redhat.com Thu Nov 21 19:02:53 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 21 Nov 2013 14:02:53 -0500 Subject: [Libvirt-cim] [PATCH] libxkutil: Added missing address element for filesystem 'disk' In-Reply-To: <1384972997-24300-1-git-send-email-mihajlov@linux.vnet.ibm.com> References: <1384972997-24300-1-git-send-email-mihajlov@linux.vnet.ibm.com> Message-ID: <528E58DD.2040007@redhat.com> On 11/20/2013 01:43 PM, Viktor Mihajlovski wrote: > This was an omission from the original patch series adding > device address support. For disks of type filesystem the addresses > were parsed however not written back. > > Signed-off-by: Viktor Mihajlovski > --- > libxkutil/xmlgen.c | 3 +++ > 1 file changed, 3 insertions(+) > ACK (and pushed) Tks, John From jferlan at redhat.com Thu Nov 21 19:59:21 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 21 Nov 2013 14:59:21 -0500 Subject: [Libvirt-cim] [PATCH] live.full_hostname: Adjust mechanism to get FQDN In-Reply-To: <528D0B2D.5060702@linux.vnet.ibm.com> References: <1384867543-8939-1-git-send-email-jferlan@redhat.com> <528D0B2D.5060702@linux.vnet.ibm.com> Message-ID: <528E6619.7010103@redhat.com> On 11/20/2013 02:19 PM, Viktor Mihajlovski wrote: > On 11/19/2013 02:25 PM, John Ferlan wrote: >> Using just socket.gethostbyaddr(socket.gethostname())[0] caused some >> issues recently in one of my DHCP testing environments. Breaking down >> the calls showed the following: >> >>>>> socket.gethostname() >> 'dhcp-186-211.bos.redhat.com' >>>>> socket.gethostbyname("dhcp-186-211.bos.redhat.com") >> '10.16.186.211' >>>>> socket.gethostbyaddr(socket.gethostname())[0] >> Traceback (most recent call last): >> File "", line 1, in >> socket.herror: [Errno 1] Unknown host >> >> While just a socket.gethostname() could have worked, using the >> socket.getfqdn() seemed to be safer just in case. >> --- >> lib/VirtLib/live.py | 7 +++++-- >> 1 file changed, 5 insertions(+), 2 deletions(-) >> >> diff --git a/lib/VirtLib/live.py b/lib/VirtLib/live.py >> index c929e71..6a10474 100644 >> --- a/lib/VirtLib/live.py >> +++ b/lib/VirtLib/live.py >> @@ -100,6 +100,9 @@ def hostname(server): >> return out >> >> def full_hostname(server): >> - """To return the fully qualifiec domain name(FQDN) of the system""" >> + """To return the fully qualified domain name(FQDN) of the system""" >> >> - return socket.gethostbyaddr(socket.gethostname())[0] >> + if socket.getfqdn().find('.') >= 0: >> + return socket.getfqdn() >> + else: >> + return socket.gethostbyaddr(socket.gethostname())[0] >> > > The server parameter seems to be entirely superfluous, > which makes me believe that the original intention might have > been something like (roughly) > > return socket.gethostbyaddr(server) > > which by itself will probably not solve the reverse lookup > issue you seem to have experienced > I think it was more of a cut-n-copy from hostname(server) just above it... I did some digging, the function does show up in the initial git entry for all of libvirt-cim, but none of the callers were using it. They were using live.hostname() passing options.ip. Then commit id '7e998246' changed the 3 callers to use full_hostname(). So how about this instead: if socket.getfqdn().find('.') >= 0: return socket.getfqdn() elif socket.gethostname().find('.') >= 0: return socket.gethostname() else: return socket.gethostbyaddr(server)[1][0] I get the following: server=localhost socket.getfqdn()=dhcp-186-211.bos.redhat.com socket.gethostname()=dhcp-186-211.bos.redhat.com socket.gethostbyaddr(server)[1][0]=localhost.localdomain >From a python prompt: >>> import socket >>> socket.gethostbyaddr("localhost") ('localhost', ['localhost.localdomain', 'localhost6', 'localhost6.localdomain6'], ['::1']) >>> Hence the [1][0] to get a FQDN rather than short name or the unexpected tuple passed back to the caller... All 3 tests will pass with the change in my configuration. I think one of the keys is whether the name is found in /etc/hosts or not. Since I use DHCP that won't happen. John From jferlan at redhat.com Thu Nov 21 20:03:33 2013 From: jferlan at redhat.com (John Ferlan) Date: Thu, 21 Nov 2013 15:03:33 -0500 Subject: [Libvirt-cim] [PATCH] RFC: Use of root/interop instead of root/PG_InterOp Message-ID: <1385064213-14142-1-git-send-email-jferlan@redhat.com> As a result of an issue I recently discovered where the most recent update to tog-pegasus on my Fedora 19 box, I have found a set of changes that will will allow my Fedora system to operate as before (mostly). The following is a link to the original discovery - there's 5 followups that are worth reading too. https://www.redhat.com/archives/libvirt-cim/2013-November/msg00008.html I'm not 100% sure the changes contained in these patches will work for all the cases, hence the request for comments as opposed to let's get this in ASAP. There may also be some extraneous adjustments - I was hoping that I could get the make install environment work, but I'm not sure I got everything right there either. At the very least the 'make rpm' will build an rpm which I can install and use. These changes do work for my Fedora 19 environment with tog-pegasus 2.12.1-8 installed. One innocent bystander in all this is the FilterList. It happened to use 'InstanceID' from the base/parent class CIM_ManagedElement. If I've read the description correctly here: http://schemas.dmtf.org/wbem/cim-html/2.34.0/CIM_ManagedElement.html The usage of InstanceID for libvirt-cim would need to go through some sort of renaming within the MOF, but it wasn't really clear "how" that would be accomplished. As an alternative, I created an InstanceUUID property which does work for me, although I'm not sure it's the right solution. The downside to doing this is that the cimtest (and I'm sure any other libvirt-cim client) requires a change to grab the FilterList data from the new different property. This creates an incompatibility situation which I'm not in favor of. Hopefully someone knows the right incantation to tell the FilterList.mof that we're going to use InstanceID --- Makefile.am | 29 ++++++++++++++++++++++++----- libvirt-cim.spec.in | 27 ++++++++++++++++++++++----- provider-register.sh | 18 +++++++++++++++++- schema/FilterList.mof | 4 ++++ src/Virt_FilterList.c | 2 +- 5 files changed, 68 insertions(+), 12 deletions(-) diff --git a/Makefile.am b/Makefile.am index 9e8e96b..69b65cf 100644 --- a/Makefile.am +++ b/Makefile.am @@ -78,6 +78,9 @@ INTEROP_MOFS = \ $(top_srcdir)/schema/ReferencedProfile.mof \ $(top_srcdir)/schema/AllocationCapabilities.mof +# The PGINTEROP_MOFS are used by tog-pegasus up through version 2.12.1 +# If support for versions prior to 2.12.1 is removed, then these defs +# can go away PGINTEROP_MOFS = \ $(top_srcdir)/schema/RegisteredProfile.mof \ $(top_srcdir)/schema/ElementConformsToProfile.mof \ @@ -157,6 +160,9 @@ INTEROP_REGS = \ $(top_srcdir)/schema/ElementConformsToProfile.registration \ $(top_srcdir)/schema/ReferencedProfile.registration +# The PGINTEROP_REGS are used by tog-pegasus up through version 2.12.1 +# If support for versions prior to 2.12.1 is removed, then these defs +# can go away PGINTEROP_REGS = \ $(top_srcdir)/schema/RegisteredProfile.registration \ $(top_srcdir)/schema/ElementConformsToProfile.registration \ @@ -181,7 +187,8 @@ EXTRA_DIST = schema $(MOFS) $(REGS) $(INTEROP_MOFS) $(INTEROP_REGS) \ .changeset .revision \ examples/diskpool.conf -# If Pegasus isn't the CIMOM target, then remove the PG_InterOp namespace from the appropriate files +# If Pegasus isn't the CIMOM target, then remove the PG_InterOp namespace +# from the appropriate files install-data-local: $(mkinstalldirs) "$(DESTDIR)$(pkgdatadir)" $(install_sh_DATA) -t "$(DESTDIR)$(pkgdatadir)" $(MOFS) @@ -189,11 +196,12 @@ install-data-local: $(install_sh_DATA) -t "$(DESTDIR)$(pkgdatadir)" $(INTEROP_MOFS) $(install_sh_DATA) -t "$(DESTDIR)$(pkgdatadir)" $(INTEROP_REGS) if [[ @CIMSERVER@ != pegasus ]]; then \ - sed -i '/^# --/,/^# --!/d' $(subst $(top_srcdir)/schema,$(DESTDIR)$(pkgdatadir), $(PGINTEROP_REGS)); \ + sed -i '/^# --/,/^# --!/d' $(subst $(top_srcdir)/schema,$(DESTDIR)$(pkgdatadir), $(PGINTEROP_REGS)); \ + sed -i '/^# --/,/^# --!/d' $(subst $(top_srcdir)/schema,$(DESTDIR)$(pkgdatadir), $(PGINTEROP_MOFS)); \ fi uninstall-local: - @list='$(MOFS) $(REGS) $(INTEROP_MOFS) $(INTEROP_REGS)'; \ + @list='$(MOFS) $(REGS) $(INTEROP_MOFS) $(INTEROP_REGS) $(PGINTEROP_REGS) $(PGINTEROP_MOFS)'; \ for p in $$list; do \ f=`echo "$$p" | sed 's|^.*/||;'`; \ echo " rm -f '$(DESTDIR)$(pkgdatadir)/$$f'"; \ @@ -209,8 +217,19 @@ postinstall: $(SHELL) provider-register.sh -v -t @CIMSERVER@ -n @CIM_VIRT_NS@ -r $(subst $(top_srcdir)/schema,$(pkgdatadir), $(REGS)) -m $(subst $(top_srcdir)/schema,$(pkgdatadir), $(MOFS)) $(SHELL) provider-register.sh -v -t @CIMSERVER@ -n root/interop -r $(subst $(top_srcdir)/schema,$(pkgdatadir), $(INTEROP_REGS)) -m $(subst $(top_srcdir)/schema,$(pkgdatadir), $(INTEROP_MOFS)) $(SHELL) provider-register.sh -v -t @CIMSERVER@ -n root/cimv2 -r $(subst $(top_srcdir)/schema,$(pkgdatadir), $(CIMV2_REGS)) -m $(subst $(top_srcdir)/schema,$(pkgdatadir), $(CIMV2_MOFS)) + # + # We need to check the version - if we're not yet at 2.12.1, then + # we'll register at root/PG_InterOp; otherwise, using just the above + # registration should be sufficient. The actual cutoff root/PG_InterOp + # not being valid was 2.12.1-5; however, --version doesn't give us that + # level of detail. The Pegasus docs imply that usage of root/interop was + # valid as of 2.12.0. + # if [[ @CIMSERVER@ = pegasus ]]; then \ - $(SHELL) provider-register.sh -v -t @CIMSERVER@ -n root/PG_InterOp -r $(subst $(top_srcdir)/schema,$(pkgdatadir), $(PGINTEROP_REGS)) -m $(subst $(top_srcdir)/schema,$(pkgdatadir), $(PGINTEROP_MOFS)); \ + CIMVER=`@CIMSERVER@ --version | awk -F. '{printf("%02d%02d%02d\n", $1,$2,$3); }'` \ + if [[ $CIMVER -lt 021201 ]]; then \ + $(SHELL) provider-register.sh -v -t @CIMSERVER@ -n root/PG_InterOp -r $(subst $(top_srcdir)/schema,$(pkgdatadir), $(PGINTEROP_REGS)) -m $(subst $(top_srcdir)/schema,$(pkgdatadir), $(PGINTEROP_MOFS)); \ + fi \ fi virsh -v | grep -q '^0.3' && cp examples/diskpool.conf $(DISK_POOL_CONFIG) || true mkdir -p $(INFO_STORE) @@ -220,7 +239,7 @@ preuninstall: $(SHELL) provider-register.sh -v -d -t @CIMSERVER@ -n root/interop -r $(subst $(top_srcdir)/schema,$(pkgdatadir), $(INTEROP_REGS)) -m $(subst $(top_srcdir)/schema,$(pkgdatadir), $(INTEROP_MOFS)) $(SHELL) provider-register.sh -v -d -t @CIMSERVER@ -n root/cimv2 -r $(subst $(top_srcdir)/schema,$(pkgdatadir), $(CIMV2_REGS)) -m $(subst $(top_srcdir)/schema,$(pkgdatadir), $(CIMV2_MOFS)) if [[ @CIMSERVER@ = pegasus ]]; then \ - $(SHELL) provider-register.sh -v -d -t @CIMSERVER@ -n root/PG_InterOp -r $(subst $(top_srcdir)/schema,$(pkgdatadir), $(PGINTEROP_REGS)) -m $(subst $(top_srcdir)/schema,$(pkgdatadir), $(PGINTEROP_MOFS)); \ + $(SHELL) provider-register.sh -v -d -t @CIMSERVER@ -n root/PG_InterOp -r $(subst $(top_srcdir)/schema,$(pkgdatadir), $(PGINTEROP_REGS)) -m $(subst $(top_srcdir)/schema,$(pkgdatadir), $(PGINTEROP_MOFS)); \ fi rpm: clean diff --git a/libvirt-cim.spec.in b/libvirt-cim.spec.in index c96451b..0eab663 100644 --- a/libvirt-cim.spec.in +++ b/libvirt-cim.spec.in @@ -199,6 +199,10 @@ rm -fr $RPM_BUILD_ROOT %{_datadir}/%{name}/ReferencedProfile.mof \\\ %{_datadir}/%{name}/AllocationCapabilities.mof +# NOTE: As of Pegasus 2.12.1-5, using root/PG_InterOp will no longer be +# valid. All mofs can just compile into root/interop. However, we +# need to keep these here for 'historical purposes'. +# %define PGINTEROP_REG %{_datadir}/%{name}/RegisteredProfile.registration \\\ %{_datadir}/%{name}/ElementConformsToProfile.registration \\\ %{_datadir}/%{name}/ReferencedProfile.registration @@ -267,12 +271,12 @@ fi %if 0%{?fedora} >= 17 || 0%{?rhel} >= 7 if [ "`systemctl is-active tog-pegasus.service 2> /dev/null`" = "active" ] then - systemctl restart tog-pegasus.service + systemctl restart tog-pegasus.service > /dev/null 2>&1 fi if [ "`systemctl is-active sblim-sfcb.service 2> /dev/null`" = "active" ] then - systemctl restart sblim-sfcb.service + systemctl restart sblim-sfcb.service > /dev/null 2>&1 fi %else /etc/init.d/tog-pegasus condrestart @@ -286,9 +290,22 @@ then %{_datadir}/%{name}/provider-register.sh -t pegasus \ -n root/interop \ -r %{INTEROP_REG} -m %{INTEROP_MOF} -v >/dev/null 2>&1 || true - %{_datadir}/%{name}/provider-register.sh -t pegasus \ - -n root/PG_InterOp \ - -r %{PGINTEROP_REG} -m %{PGINTEROP_MOF} -v >/dev/null 2>&1 || true + # + # We need to check the version - if we're not yet at 2.12.1, then + # we'll register at root/PG_InterOp; otherwise, using just the above + # registration should be sufficient. The actual cutoff root/PG_InterOp + # not being valid was 2.12.1-5; however, --version doesn't give us that + # level of detail. The Pegasus docs imply that usage of root/interop was + # valid as of 2.12.0. + # + CIMVER=`/usr/sbin/cimserver --version | \ + awk -F. '{printf("%02d%02d%02d\n", $1,$2,$3); }'` + if [ $CIMVER -lt 021201 ] + then + %{_datadir}/%{name}/provider-register.sh -t pegasus \ + -n root/PG_InterOp \ + -r %{PGINTEROP_REG} -m %{PGINTEROP_MOF} -v >/dev/null 2>&1 || true + fi %{_datadir}/%{name}/provider-register.sh -t pegasus \ -n root/cimv2\ -r %{CIMV2_REG} -m %{CIMV2_MOF} -v >/dev/null 2>&1 || true diff --git a/provider-register.sh b/provider-register.sh index abe8e95..f66fe54 100755 --- a/provider-register.sh +++ b/provider-register.sh @@ -274,7 +274,23 @@ pegasus_install() chatter Registering providers with $state cimserver '('$version')' chatter Installing mofs into namespace $namespace from path $mofpath $CIMMOF -uc -I $mofpath -n $namespace $mymofs && - $CIMMOF -uc -n root/PG_Interop $_REGFILENAME + # + # If compare_version returns false here (e.g. $version is less than + # "2.12.1", then we will compile into root/PG_InterOp; otherwise, + # compile into root/interop. As of 2.12.1-5 using the PG_InterOp + # will fail. Since we cannot get that level of detail out of the + # --version output, "assume" that 2.12.1 -> 2.12.1-4 will be able + # to use the new namespace. The Pegasus docs imply as of 2.12.0 using + # root/interop was preferred. + # + if compare_version "$version" "2.12.1" + then + chatter Installing $_REGFILENAME into root/PG_InterOp + $CIMMOF -uc -n root/PG_Interop $_REGFILENAME + else + chatter Installing $_REGFILENAME into root/interop + $CIMMOF -uc -n root/interop $_REGFILENAME + fi else echo "Failed to build pegasus registration MOF." >&2 return 1 diff --git a/schema/FilterList.mof b/schema/FilterList.mof index 7339db6..b7d8551 100644 --- a/schema/FilterList.mof +++ b/schema/FilterList.mof @@ -10,4 +10,8 @@ class KVM_FilterList : CIM_FilterList MinValue(-1000), MaxValue(1000)] sint16 Priority = 500; + + [Description("The network filter UUID value. This value was formerly " + "stored in the InstanceID field.")] + string InstanceUUID; }; diff --git a/src/Virt_FilterList.c b/src/Virt_FilterList.c index b248004..6a9112e 100644 --- a/src/Virt_FilterList.c +++ b/src/Virt_FilterList.c @@ -76,7 +76,7 @@ static CMPIInstance *convert_filter_to_instance( CMSetProperty(inst, "SystemName", sys_name, CMPI_chars); CMSetProperty(inst, "SystemCreationClassName", sys_ccname, CMPI_chars); CMSetProperty(inst, "Name", (CMPIValue *)filter->name, CMPI_chars); - CMSetProperty(inst, "InstanceID", (CMPIValue *)filter->uuid, + CMSetProperty(inst, "InstanceUUID", (CMPIValue *)filter->uuid, CMPI_chars); CMSetProperty(inst, "Direction", (CMPIValue *)&direction, CMPI_uint16); -- 1.8.3.1 From fiuczy at linux.vnet.ibm.com Fri Nov 22 16:18:13 2013 From: fiuczy at linux.vnet.ibm.com (Boris Fiuczynski) Date: Fri, 22 Nov 2013 17:18:13 +0100 Subject: [Libvirt-cim] [PATCH 00/20] REWORK/PARIAL: Changes to solve unsupported tag issue In-Reply-To: <528D7AE1.3000202@linux.vnet.ibm.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> <528A2B58.5010907@linux.vnet.ibm.com> <528BDCF9.8010109@redhat.com> <528CB8A5.6090702@linux.vnet.ibm.com> <528D2537.3010408@redhat.com> <528D7AE1.3000202@linux.vnet.ibm.com> Message-ID: <528F83C5.2060108@linux.vnet.ibm.com> On 11/21/2013 04:15 AM, Xu Wang wrote: > > ? 2013/11/21 5:10, John Ferlan ??: >> On 11/20/2013 08:27 AM, Boris Fiuczynski wrote: >>> John and Xu, >>> >>> On 11/19/2013 10:49 PM, John Ferlan wrote: >>>> On 11/18/2013 09:59 AM, Boris Fiuczynski wrote: >>>>> John and Xu Wang, >>>>> here are a few general observations from side: >>>> First off - I tend to find myself agreeing with Boris here. I think the >>>> concept is important and necessary; however, I'm not convinced the >>>> implementation goes far enough. >>>> >>>>> 1) I agree that it makes sense to preserve the unknown xml "entities" >>>>> even so it can create complex scenarios and even new kinds of >>>>> errors if >>>>> unknown entities depend on known entities which get modified making >>>>> them >>>>> unusable for the unknown entities. This error would probably be the >>>>> reversal of what is currently the problem when unknown entities simply >>>>> disappear. >>>> Is there a more concrete example of "new kinds of errors if unknown >>>> entities depend on known entities which get modified making them >>>> unusable for the unknown entities" that can be given? Just for >>>> clarity. >>>> I've read that line a few times and I'm still not sure :-) >>> OK, let's take a look at device type disk. >>> Since 1.0.2 the property sgio was added. Let's assume this is the >>> unknown entity. sgio is only valid for property device "lun". >>> If one now changes the property device to "disk" than the unknown entity >>> sgio would cause an error when specified. >>> >> Ah - I see. Not only do you have to manage the properties you have to >> know how to use them as well and all their rules. I forgot about that. I >> came from HP/HPVM and yes, this brings back all sorts of memories... >> >> Seems like in this case, when/if the property was changed from "lun" to >> "disk" - code would have to search that 'others' list for the "sgio" >> property and know how to handle adjusting it. That'll get tricky... You cannot search in others for something you do not know about... unless you can code magic! :-) > In fact there is lots of hacking code like this in libvirt-cim. > > My suggestion is, if some node was changed (such as "lun" to "disk"), > its unknown > sub-entities should be marked as INACTIVE to avoid that issue. That is, > we couldn't > keep the validity of unknown entities, so maybe let libvirt-cim just do > like now is a > better choice. > > If you think sometimes the idea above still could miss something useful, > some hack > code need to be added to handle issue like this. The problem here is actually that the dependency is new and the old code does not know about it. The old libvirt-cim pattern was getting around this trouble by just maintaining what is known and neglect the unknown. Xu is trying to prevent the neglecting of tags and properties with his patch. With this in mind I would find it strange to break this new pattern now by starting to remove/invalidate unknown tag/properties. The trouble with the new pattern is that the error causing tags/properties are outside of the libvirt-cim management scope and cannot be adjust via libvirt-cim to allow modifications of the domain in this case. I think that the errors that are caused in this situations are the reversal of errors caused by the current pattern of "neglect the unknown". I do not know if it would be a good idea to mix these patterns up. Looking at the mixed case from the users perspective I currently suppose that most users would proably call it a bug if tags would mostly be maintained and sometimes not. Even if the mixed case would be used: which known tags/properties change would trigger the invalidation/deletion of what unknown sub tags/properties and to which tree depth? What happens when relationships across device instances arise? We really need to be cautious of what we are doing here. Just an idea (while talking about users): Should the user perhaps be made capable to choose the "mode" of libvirt-cim regarding the handling of unknown tags & properties? If we would do that the freedom of how the new mode can behave increases. I know that this would mean a rather large change but I did not want to let this opportunity pass without mentioning it. I am also aware that this would double the testing efforts and likely increase the coding efforts for the future. >>>> ... >> >>> In addition I would like to suggest that for easier access into the new >>> tree structure a reference to the corresponding tree node element should >>> be stored as "other" (maybe rename it to node_ref) in the existing >>> libvirt-cim data structure. >> That works too (rather than borrowing and returning). > That's a good idea. I think using 'strdup()' less could save memory space > and running time. It is likely to save also some write back operations in step 4./5. outline by Xu below. >> >>> A question comes to my mind about the usage of one tree: Does it create >>> a problem for the helper methods (seek..., fetch..., ...) on instances >>> that are not leafs (e.g. domain). I guess it would now be needed to know >>> the hierarchy depth at which these methods stop. That is something Xu is >>> the best to answer. >>> >> Right! and a bit of prototyping... Validate the algorithm/thoughts work >> with 'domain' and perhaps 'file'/'block'... >> ... >>>> I think we need to come up with an agreement on the architecture first >>>> (currently patches 1-3 and then what was the original 21/48). >>>> >>>> Those should be submitted first without any other patches and without >>>> necessarily being called except perhaps through xml_parse_test which >>>> should be used to prove that the "new" parsing and generation does no >>>> worse than the old (a 5th patch perhaps). >>> Good point! The xml_parse_test is a good validation tool for the first >>> part. >>>> Once those patches are in, then submit the patches that read/write >>>> domain level data... Then submit the patches that read/write domain/mem >>>> data... then domain/vcpu, etc. Also, rather than separate the xml >>>> parsing & generating, keep it together for each type. >>> This seems like a valid and doable approach. Do you agree as well, Xu? > I can't agree with you more:-) Looks like we all agree on this. :-) >>> > And the whole process like this? > > 1. Traverse content of xml -> build references -> save into 'others' > (just name it temporarily) > 2. Fetch/handle data into virt_device structure > 3. Resource operations (just like original, some change may need to be > done such as status mark) > 4. Restore the data in the virt_devices back to the xml "back to the xml"? Shouldn't that be "back into the 'others' structure"? > 5. Regenerate the xml based on the references (status mainly) > 6. The following steps (just like original) > > Some maros (functions) like this maybe needed, > BUILD_REFERENCES, > GET_NODE, > GET_PROP, > UPDATE_STATUS, > ...... > > Thanks, > Xu Wang >> _______________________________________________ >> Libvirt-cim mailing list >> Libvirt-cim at redhat.com >> https://www.redhat.com/mailman/listinfo/libvirt-cim >> > -- Mit freundlichen Gr??en/Kind regards Boris Fiuczynski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martina K?deritz Gesch?ftsf?hrung: Dirk Wittkopp Sitz der Gesellschaft: B?blingen Registergericht: Amtsgericht Stuttgart, HRB 243294 From cngesaint at gmail.com Tue Nov 26 12:31:16 2013 From: cngesaint at gmail.com (Xu Wang) Date: Tue, 26 Nov 2013 20:31:16 +0800 Subject: [Libvirt-cim] [PATCH 00/20] REWORK/PARIAL: Changes to solve unsupported tag issue In-Reply-To: <528F83C5.2060108@linux.vnet.ibm.com> References: <1384475049-11435-1-git-send-email-jferlan@redhat.com> <528A2B58.5010907@linux.vnet.ibm.com> <528BDCF9.8010109@redhat.com> <528CB8A5.6090702@linux.vnet.ibm.com> <528D2537.3010408@redhat.com> <528D7AE1.3000202@linux.vnet.ibm.com> <528F83C5.2060108@linux.vnet.ibm.com> Message-ID: <52949494.1040805@gmail.com> ? 2013/11/23 0:18, Boris Fiuczynski ??: > On 11/21/2013 04:15 AM, Xu Wang wrote: >> >> ? 2013/11/21 5:10, John Ferlan ??: >>> On 11/20/2013 08:27 AM, Boris Fiuczynski wrote: >>>> John and Xu, >>>> >>>> On 11/19/2013 10:49 PM, John Ferlan wrote: >>>>> On 11/18/2013 09:59 AM, Boris Fiuczynski wrote: >>>>>> John and Xu Wang, >>>>>> here are a few general observations from side: >>>>> First off - I tend to find myself agreeing with Boris here. I >>>>> think the >>>>> concept is important and necessary; however, I'm not convinced the >>>>> implementation goes far enough. >>>>> >>>>>> 1) I agree that it makes sense to preserve the unknown xml >>>>>> "entities" >>>>>> even so it can create complex scenarios and even new kinds of >>>>>> errors if >>>>>> unknown entities depend on known entities which get modified making >>>>>> them >>>>>> unusable for the unknown entities. This error would probably be the >>>>>> reversal of what is currently the problem when unknown entities >>>>>> simply >>>>>> disappear. >>>>> Is there a more concrete example of "new kinds of errors if unknown >>>>> entities depend on known entities which get modified making them >>>>> unusable for the unknown entities" that can be given? Just for >>>>> clarity. >>>>> I've read that line a few times and I'm still not sure :-) >>>> OK, let's take a look at device type disk. >>>> Since 1.0.2 the property sgio was added. Let's assume this is the >>>> unknown entity. sgio is only valid for property device "lun". >>>> If one now changes the property device to "disk" than the unknown >>>> entity >>>> sgio would cause an error when specified. >>>> >>> Ah - I see. Not only do you have to manage the properties you have to >>> know how to use them as well and all their rules. I forgot about >>> that. I >>> came from HP/HPVM and yes, this brings back all sorts of memories... >>> >>> Seems like in this case, when/if the property was changed from "lun" to >>> "disk" - code would have to search that 'others' list for the "sgio" >>> property and know how to handle adjusting it. That'll get tricky... > You cannot search in others for something you do not know about... > unless you can code magic! :-) If needed that's possible. We just coding just like...xpath. Before that we should make sure if it's necessary. >> In fact there is lots of hacking code like this in libvirt-cim. >> >> My suggestion is, if some node was changed (such as "lun" to "disk"), >> its unknown >> sub-entities should be marked as INACTIVE to avoid that issue. That is, >> we couldn't >> keep the validity of unknown entities, so maybe let libvirt-cim just do >> like now is a >> better choice. >> >> If you think sometimes the idea above still could miss something useful, >> some hack >> code need to be added to handle issue like this. > The problem here is actually that the dependency is new and the old > code does not know about it. The old libvirt-cim pattern was getting > around this trouble by just maintaining what is known and neglect the > unknown. > Xu is trying to prevent the neglecting of tags and properties with his > patch. With this in mind I would find it strange to break this new > pattern now by starting to remove/invalidate unknown tag/properties. > The trouble with the new pattern is that the error causing > tags/properties are outside of the libvirt-cim management scope and > cannot be adjust via libvirt-cim to allow modifications of the domain > in this case. > > I think that the errors that are caused in this situations are the > reversal of errors caused by the current pattern of "neglect the > unknown". I do not know if it would be a good idea to mix these > patterns up. Looking at the mixed case from the users perspective I > currently suppose that most users would proably call it a bug if tags > would mostly be maintained and sometimes not. > Even if the mixed case would be used: which known tags/properties > change would trigger the invalidation/deletion of what unknown sub > tags/properties and to which tree depth? > What happens when relationships across device instances arise? > > We really need to be cautious of what we are doing here. > > Just an idea (while talking about users): Should the user perhaps be > made capable to choose the "mode" of libvirt-cim regarding the > handling of unknown tags & properties? If we would do that the freedom > of how the new mode can behave increases. > I know that this would mean a rather large change but I did not want > to let this opportunity pass without mentioning it. I am also aware > that this would double the testing efforts and likely increase the > coding efforts for the future. Yeah,that's a good choice to decide behavior by option. We can keep a 'compatible mode' to those users who 'like old things'. On the other hand it's no doubt that will add the scale of code and maintaining work. But...what should be like with the new mode? Just keep the original content or add some hacking code to solve conflicts like this? > >>>>> > ... >>> >>>> In addition I would like to suggest that for easier access into the >>>> new >>>> tree structure a reference to the corresponding tree node element >>>> should >>>> be stored as "other" (maybe rename it to node_ref) in the existing >>>> libvirt-cim data structure. >>> That works too (rather than borrowing and returning). >> That's a good idea. I think using 'strdup()' less could save memory >> space >> and running time. > It is likely to save also some write back operations in step 4./5. > outline by Xu below. > >>> >>>> A question comes to my mind about the usage of one tree: Does it >>>> create >>>> a problem for the helper methods (seek..., fetch..., ...) on instances >>>> that are not leafs (e.g. domain). I guess it would now be needed to >>>> know >>>> the hierarchy depth at which these methods stop. That is something >>>> Xu is >>>> the best to answer. >>>> >>> Right! and a bit of prototyping... Validate the algorithm/thoughts >>> work >>> with 'domain' and perhaps 'file'/'block'... >>> > ... >>>>> I think we need to come up with an agreement on the architecture >>>>> first >>>>> (currently patches 1-3 and then what was the original 21/48). >>>>> >>>>> Those should be submitted first without any other patches and without >>>>> necessarily being called except perhaps through xml_parse_test which >>>>> should be used to prove that the "new" parsing and generation does no >>>>> worse than the old (a 5th patch perhaps). >>>> Good point! The xml_parse_test is a good validation tool for the first >>>> part. >>>>> Once those patches are in, then submit the patches that read/write >>>>> domain level data... Then submit the patches that read/write >>>>> domain/mem >>>>> data... then domain/vcpu, etc. Also, rather than separate the xml >>>>> parsing & generating, keep it together for each type. >>>> This seems like a valid and doable approach. Do you agree as well, Xu? >> I can't agree with you more:-) > Looks like we all agree on this. :-) > >>>> >> And the whole process like this? >> >> 1. Traverse content of xml -> build references -> save into 'others' >> (just name it temporarily) >> 2. Fetch/handle data into virt_device structure >> 3. Resource operations (just like original, some change may need to be >> done such as status mark) >> 4. Restore the data in the virt_devices back to the xml > > "back to the xml"? Shouldn't that be "back into the 'others' structure"? You're right. Maybe my original understanding isn't accurate. I repeat it. Firstly libvirt-cim read data from xml and save them into 'others' structure. Then, some data in the virt_devices just use pointer point to the data in 'others' intead of using strdup(), isn't it? Thanks, Xu Wang > >> 5. Regenerate the xml based on the references (status mainly) >> 6. The following steps (just like original) >> >> Some maros (functions) like this maybe needed, >> BUILD_REFERENCES, >> GET_NODE, >> GET_PROP, >> UPDATE_STATUS, >> ...... >> >> Thanks, >> Xu Wang >>> _______________________________________________ >>> Libvirt-cim mailing list >>> Libvirt-cim at redhat.com >>> https://www.redhat.com/mailman/listinfo/libvirt-cim >>> >> > > From fiuczy at linux.vnet.ibm.com Tue Nov 26 12:40:30 2013 From: fiuczy at linux.vnet.ibm.com (Boris Fiuczynski) Date: Tue, 26 Nov 2013 13:40:30 +0100 Subject: [Libvirt-cim] [PATCH] RFC: Use of root/interop instead of root/PG_InterOp In-Reply-To: <1385064213-14142-1-git-send-email-jferlan@redhat.com> References: <1385064213-14142-1-git-send-email-jferlan@redhat.com> Message-ID: <529496BE.4030103@linux.vnet.ibm.com> On 11/21/2013 09:03 PM, John Ferlan wrote: ... > diff --git a/schema/FilterList.mof b/schema/FilterList.mof > index 7339db6..b7d8551 100644 > --- a/schema/FilterList.mof > +++ b/schema/FilterList.mof > @@ -10,4 +10,8 @@ class KVM_FilterList : CIM_FilterList > MinValue(-1000), > MaxValue(1000)] > sint16 Priority = 500; > + > + [Description("The network filter UUID value. This value was formerly " > + "stored in the InstanceID field.")] > + string InstanceUUID; > }; > diff --git a/src/Virt_FilterList.c b/src/Virt_FilterList.c > index b248004..6a9112e 100644 > --- a/src/Virt_FilterList.c > +++ b/src/Virt_FilterList.c > @@ -76,7 +76,7 @@ static CMPIInstance *convert_filter_to_instance( > CMSetProperty(inst, "SystemName", sys_name, CMPI_chars); > CMSetProperty(inst, "SystemCreationClassName", sys_ccname, CMPI_chars); > CMSetProperty(inst, "Name", (CMPIValue *)filter->name, CMPI_chars); > - CMSetProperty(inst, "InstanceID", (CMPIValue *)filter->uuid, > + CMSetProperty(inst, "InstanceUUID", (CMPIValue *)filter->uuid, > CMPI_chars); > CMSetProperty(inst, "Direction", (CMPIValue *)&direction, CMPI_uint16); > Hi John, the changes regarding the FilterList seem strange to me. I would not expect any schema changes. Can you elaborate a bit more why these changes are needed, what made you create these changes and how could one recreate a possible error which these changes fix? -- Mit freundlichen Gr??en/Kind regards Boris Fiuczynski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Martina K?deritz Gesch?ftsf?hrung: Dirk Wittkopp Sitz der Gesellschaft: B?blingen Registergericht: Amtsgericht Stuttgart, HRB 243294 From jferlan at redhat.com Wed Nov 27 14:14:50 2013 From: jferlan at redhat.com (John Ferlan) Date: Wed, 27 Nov 2013 09:14:50 -0500 Subject: [Libvirt-cim] [PATCH] RFC: Use of root/interop instead of root/PG_InterOp In-Reply-To: <529496BE.4030103@linux.vnet.ibm.com> References: <1385064213-14142-1-git-send-email-jferlan@redhat.com> <529496BE.4030103@linux.vnet.ibm.com> Message-ID: <5295FE5A.6090200@redhat.com> On 11/26/2013 07:40 AM, Boris Fiuczynski wrote: > On 11/21/2013 09:03 PM, John Ferlan wrote: > ... > >> diff --git a/schema/FilterList.mof b/schema/FilterList.mof >> index 7339db6..b7d8551 100644 >> --- a/schema/FilterList.mof >> +++ b/schema/FilterList.mof >> @@ -10,4 +10,8 @@ class KVM_FilterList : CIM_FilterList >> MinValue(-1000), >> MaxValue(1000)] >> sint16 Priority = 500; >> + >> + [Description("The network filter UUID value. This value was >> formerly " >> + "stored in the InstanceID field.")] >> + string InstanceUUID; >> }; >> diff --git a/src/Virt_FilterList.c b/src/Virt_FilterList.c >> index b248004..6a9112e 100644 >> --- a/src/Virt_FilterList.c >> +++ b/src/Virt_FilterList.c >> @@ -76,7 +76,7 @@ static CMPIInstance *convert_filter_to_instance( >> CMSetProperty(inst, "SystemName", sys_name, CMPI_chars); >> CMSetProperty(inst, "SystemCreationClassName", sys_ccname, >> CMPI_chars); >> CMSetProperty(inst, "Name", (CMPIValue *)filter->name, >> CMPI_chars); >> - CMSetProperty(inst, "InstanceID", (CMPIValue *)filter->uuid, >> + CMSetProperty(inst, "InstanceUUID", (CMPIValue *)filter->uuid, >> CMPI_chars); >> CMSetProperty(inst, "Direction", (CMPIValue *)&direction, >> CMPI_uint16); >> > Hi John, > the changes regarding the FilterList seem strange to me. I would not > expect any schema changes. > Can you elaborate a bit more why these changes are needed, what made you > create these changes and how could one recreate a possible error which > these changes fix? > BTW: I had a very bad disk crash in my main work environment - I've lost everything... It's going to take me a few days to recover... Unfortunately for me the backups I thought were occurring - weren't... The following is from memory and some notes I have written down... It seems the newer environment doesn't allow setting the "InstanceID" property. That property is a member of CIM_ManagedElement and the link I pointed at in my RFC describes the field. I'm assuming that since we've moved to a more common "root/interop" environment and away from the (more private) "root/PG_InterOp" environment that somehow there needs to be a way to more uniquely identify which provider (namespace) is adjusting that value. Reading that webpage description really didn't help me figure out what the magic incantation might need to be. So as an alternative I created the new property and thus why I called this an RFC because - quite frankly I wasn't sure how to handle it. Note that in the new environment "wbemcli ei http://root:password at localhost/root/virt:KVM_FilterList" failed to show the InstanceID parameter (assuming of course you've applied the rest of the changes to use "root/interop")... Because InstanceID doesn't exist the cimtest for FilterList will fail when accessing 'InstanceID' in the python dictionary/list (damn, I wish I'd sent that one too - now I have to recreate - sigh) John