[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[Rdo-list] issue with numa and cpu pinning using SRIOV ports



Hi all,

I have a rdo kilo deployment, using sr-iov ports to my instances. I'm trying to configure NUMA topology and CPU pinning for some telco based workloads based on this doc: http://redhatstackblog.redhat.com/2015/05/05/cpu-pinning-and-numa-topology-awareness-in-openstack-compute/
I have 3 compute nodes, I'm trying to use one of them to use cpu pinning.
I've configured it like this:
Compute Node (total 24 cpus)

/etc/nova/nova.conf
vcpu_pin_set=2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,22,23

Changed grub to isolate my cpus:
#grubby --update-kernel=ALL --args="isolcpus=2,3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,22,23"
#grub2-install /dev/sda
Controller Nodes: /etc/nova/nova.conf scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,PciPassthroughFilter,NUMATopologyFilter,AggregateInstanceExtraSpecsFilter scheduler_available_filters = nova.scheduler.filters.all_filters scheduler_available_filters = nova.scheduler.filters.pci_passthrough_filter.PciPassthroughFilter Created host aggregate performance #nova aggregate-create performance #nova aggregate-set-metadata 1 pinned=true
#nova aggregate-add-host 1 compute03

Created host aggregate normal
#nova aggregate-create normal
#nova aggregate-set-metadata 2 pinned=false
#nova aggregate-add-host 2 compute01
#nova aggregate-add-host 2 compute02
Created the flavor with cpu pinning #nova flavor-create m1.performance 6 2048 20 4 #nova flavor-key 6 set hw:cpu_policy=dedicated #nova flavor-key 6 set aggregate_instance_extra_specs:pinned=true The issue is: With SR-IOV ports it only let's me create instances with 6 vcpus in total with the conf described above. Without SR-IOV, using OVS, I don't have that limitation. Is this a bug or something? I've seen this: https://bugs.launchpad.net/nova/+bug/1441169, however I have the patch, and as I said it works for the first 6 vcpus with my configuration.
Some relevant logs:
/var/log/nova/nova-scheduler.log
2015-11-06 11:18:17.955 59494 DEBUG nova.filters [req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d 9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - - -] Starting with 3 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:70
2015-11-06 11:18:17.955 59494 DEBUG nova.filters [req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d 9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - - -] Filter RetryFilter returned 3 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.955 59494 DEBUG nova.filters [req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d 9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - - -] Filter AvailabilityZoneFilter returned 3 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.955 59494 DEBUG nova.filters [req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d 9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - - -] Filter RamFilter returned 3 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.956 59494 DEBUG nova.filters [req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d 9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - - -] Filter ComputeFilter returned 3 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.956 59494 DEBUG nova.filters [req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d 9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - - -] Filter ComputeCapabilitiesFilter returned 3 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.956 59494 DEBUG nova.filters [req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d 9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - - -] Filter ImagePropertiesFilter returned 3 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.956 59494 DEBUG nova.filters [req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d 9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - - -] Filter ServerGroupAntiAffinityFilter returned 3 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.956 59494 DEBUG nova.filters [req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d 9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - - -] Filter ServerGroupAffinityFilter returned 3 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.957 59494 DEBUG nova.filters [req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d 9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - - -] Filter PciPassthroughFilter returned 3 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:84
2015-11-06 11:18:17.959 59494 DEBUG nova.filters [req-9e20f8a9-384f-45c2-aa99-2d7b3100c98d 9340dc4e70a14aeb82013e5a1631de80 d5ecb0eea96f4996b565fd983a768b11 - - -] Filter NUMATopologyFilter returned 2 host(s) get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:84
Any help would be appreciated.

Thanks,
Pedro Sousa

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]