Skip to content
  • Alexey Kardashevskiy's avatar
    vfio-pci, ppc64/spapr: Reorder group-to-container attaching · 8c37faa4
    Alexey Kardashevskiy authored
    
    
    At the moment VFIO PCI device initialization works as follows:
    vfio_realize
    	vfio_get_group
    		vfio_connect_container
    			register memory listeners (1)
    			update QEMU groups lists
    		vfio_kvm_device_add_group
    
    Then (example for pseries) the machine reset hook triggers region_add()
    for all regions where listeners from (1) are listening:
    
    ppc_spapr_reset
    	spapr_phb_reset
    		spapr_tce_table_enable
    			memory_region_add_subregion
    				vfio_listener_region_add
    					vfio_spapr_create_window
    
    This scheme works fine until we need to handle VFIO PCI device hotplug
    and we want to enable PPC64/sPAPR in-kernel TCE acceleration on,
    i.e. after PCI hotplug we need a place to call
    ioctl(vfio_kvm_device_fd, KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE).
    Since the ioctl needs a LIOBN fd (from sPAPRTCETable) and a IOMMU group fd
    (from VFIOGroup), vfio_listener_region_add() seems to be the only place
    for this ioctl().
    
    However this only works during boot time because the machine reset
    happens strictly after all devices are finalized. When hotplug happens,
    vfio_listener_region_add() is called when a memory listener is registered
    but when this happens:
    1. new group is not added to the container->group_list yet;
    2. VFIO KVM device is unaware of the new IOMMU group.
    
    This moves bits around to have all necessary VFIO infrastructure
    in place for both initial startup and hotplug cases.
    
    [aw: ie, register vfio groups with kvm prior to memory listener
    registration such that kvm-vfio pseudo device ioctls are available
    during the region_add callback]
    
    Signed-off-by: default avatarAlexey Kardashevskiy <aik@ozlabs.ru>
    Reviewed-by: default avatarDavid Gibson <david@gibson.dropbear.id.au>
    Signed-off-by: default avatarAlex Williamson <alex.williamson@redhat.com>
    8c37faa4