[libvirt] virsh nodeinfo CPU detection questions

Peter Krempa pkrempa at redhat.com
Tue Feb 5 11:34:09 UTC 2013


On 01/30/13 23:12, Doug Goldstein wrote:
> On Wed, Jan 30, 2013 at 12:53 PM, Scott Sullivan
> <ssullivan at liquidweb.com> wrote:
>> On 01/29/2013 02:55 PM, Doug Goldstein wrote:
>>>
>>> On Tue, Jan 29, 2013 at 9:52 AM, Scott Sullivan<ssullivan at liquidweb.com>
>>> wrote:
>>>
>>>>
>>>> I have a machine with four AMD Opteron(TM) 6272 processors:
>>>>
>>>>
>>>> http://www.amd.com/us/products/server/processors/6000-series-platform/6200/Pages/6200-series-processors.aspx#5
>>>>
>>>> I use 'virsh nodeinfo' to determine the amount of physical CPUs, and CPU
>>>> cores. What values should I be looking at in the output of nodeinfo to
>>>> determine this? With 1.X libvirts the output values have changed from
>>>> 0.9.4:
>>>>
>>>> 0.9.4:
>>>> [root at host ~]# virsh nodeinfo
>>>> CPU model: x86_64
>>>> CPU(s): 64
>>>> CPU frequency: 2099 MHz
>>>> CPU socket(s): 4
>>>> Core(s) per socket: 8
>>>> Thread(s) per core: 2
>>>> NUMA cell(s): 1
>>>> Memory size: 24713980 kB
>>>> [root at host28 ~]#
>>>>
>>>> 1.0.1:
>>>> [root at host ~]# virsh nodeinfo
>>>> CPU model: x86_64
>>>> CPU(s): 64
>>>> CPU frequency: 2099 MHz
>>>> CPU socket(s): 1
>>>> Core(s) per socket: 8
>>>> Thread(s) per core: 2
>>>> NUMA cell(s): 4
>>>> Memory size: 24713980 KiB
>>>> [root at host28 ~]#
>>>>
>>>> As you can see, the CPU socket and NUMA cells values have changed in
>>>> these
>>>> two versions, on the same machine. To get the physical CPU count, is it
>>>> safe
>>>> to assume this?
>>>>
>>>> Numa cell(s) X CPU Socket(s) == physical CPUs present
>>>>
>>>> However, I am not positive what formula I should use to determine the CPU
>>>> core count. On this processor, it has 16 cores. Would that mean I can do
>>>> this?
>>>>
>>>> Core(s) per socket X Thread(s) per core: == CPU cores
>>>>
>>>> Thanks in advance for clarifying.
>>>>
>>>> --
>>>> libvir-list mailing list
>>>> libvir-list at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/libvir-list
>>>>
>>>
>>> Scott,
>>>
>>> I've got the exact same CPUs (Dell PowerEdge R815). You'll find the
>>> NUMA info is detected node info is detected incorrectly. Oddly we have
>>> different data detected. What kernel are you running?
>>>
>>> with libvirt 1.0.1
>>> root at host ~ # virsh nodeinfo
>>> CPU model:           x86_64
>>> CPU(s):              64
>>> CPU frequency:       2100 MHz
>>> CPU socket(s):       1
>>> Core(s) per socket:  64
>>> Thread(s) per core:  1
>>> NUMA cell(s):        1
>>> Memory size:         132013200 KiB
>>>
>>> virsh capabilities is a bit "smarter" but still not 100%. It sees 8
>>> NUMA nodes and puts the CPUs in the correct nodes but still detects
>>> the CPU incorrectly.
>>>
>>>
>>
>> Doug,
>>
>> I'm running a RedHat kernel; 2.6.32-220.2.1.el6.x86_64 specifically.
>>
>> Thanks for the insight of your experiences, very interesting. I find the
>> same that the NUMA info appears detected incorrectly for this processor.
>
> Scott,
>
> It appears that the CPU info isn't detected correctly because it can't
> be detected correctly or really expressed in the values we know/have.
> For some info you can see:
> https://bugzilla.redhat.com/show_bug.cgi?id=874050
>
> The issue is that libvirt and to extent the Linux kernel expect 1
> physical socket to equate to 1 NUMA node. In the case of these
> processors there are really 2 individual "packages" in 1 physical
> socket. Each package has its own memory controller and that is tied to
> 8 cores. Hence how you get 8 NUMA nodes with 8 processors each.
> However to add to the confusion AMD has implemented their own type of
> hyperthreading in that they have two separate cores but 2 cores share
> 1 FPU. So in reality each package has 8 cores but if you're doing FPU
> work, you only have 4 per package.
>
> The previous decision was that 4 sockets, 8 cores per socket and 2
> thread per core was not a correct way to express this CPU since it is
> true that each core separate. But the current solution is IMHO worse
> by saying 1 socket, 64 cores, 1 thread per core. A better result would
> be to say 8 sockets, 8 cores per socket, and 1 thread per core. But
> that would require us to break the idea that a socket is a physical
> socket and align a socket with a "package".

The data in the "nodeinfo" structure (retrirevable by "virsh nodeinfo") 
have to add up to the total number of processors in the host if you 
multiply all the fields. This is due to historic reasons.

>
> Anyone else thoughts? Would this be acceptable? Right now the output
> is broken and if you put any NUMA values into the XML or use numad the
> domain will not start and will give a cryptic error message. Which is
> a lot worst than reporting 4 sockets, 8 cores, 1 thread or 8 sockets,
> 8 cores and 1 thread IMHO.

To get correct data, you have to use libvirt-1.0.2 with this script:

https://bugzilla.redhat.com/attachment.cgi?id=690965&action=edit


>




More information about the libvir-list mailing list