[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [libvirt] virsh nodeinfo CPU detection questions



On Wed, Jan 30, 2013 at 12:53 PM, Scott Sullivan
<ssullivan liquidweb com> wrote:
> On 01/29/2013 02:55 PM, Doug Goldstein wrote:
>>
>> On Tue, Jan 29, 2013 at 9:52 AM, Scott Sullivan<ssullivan liquidweb com>
>> wrote:
>>
>>>
>>> I have a machine with four AMD Opteron(TM) 6272 processors:
>>>
>>>
>>> http://www.amd.com/us/products/server/processors/6000-series-platform/6200/Pages/6200-series-processors.aspx#5
>>>
>>> I use 'virsh nodeinfo' to determine the amount of physical CPUs, and CPU
>>> cores. What values should I be looking at in the output of nodeinfo to
>>> determine this? With 1.X libvirts the output values have changed from
>>> 0.9.4:
>>>
>>> 0.9.4:
>>> [root host ~]# virsh nodeinfo
>>> CPU model: x86_64
>>> CPU(s): 64
>>> CPU frequency: 2099 MHz
>>> CPU socket(s): 4
>>> Core(s) per socket: 8
>>> Thread(s) per core: 2
>>> NUMA cell(s): 1
>>> Memory size: 24713980 kB
>>> [root host28 ~]#
>>>
>>> 1.0.1:
>>> [root host ~]# virsh nodeinfo
>>> CPU model: x86_64
>>> CPU(s): 64
>>> CPU frequency: 2099 MHz
>>> CPU socket(s): 1
>>> Core(s) per socket: 8
>>> Thread(s) per core: 2
>>> NUMA cell(s): 4
>>> Memory size: 24713980 KiB
>>> [root host28 ~]#
>>>
>>> As you can see, the CPU socket and NUMA cells values have changed in
>>> these
>>> two versions, on the same machine. To get the physical CPU count, is it
>>> safe
>>> to assume this?
>>>
>>> Numa cell(s) X CPU Socket(s) == physical CPUs present
>>>
>>> However, I am not positive what formula I should use to determine the CPU
>>> core count. On this processor, it has 16 cores. Would that mean I can do
>>> this?
>>>
>>> Core(s) per socket X Thread(s) per core: == CPU cores
>>>
>>> Thanks in advance for clarifying.
>>>
>>> --
>>> libvir-list mailing list
>>> libvir-list redhat com
>>> https://www.redhat.com/mailman/listinfo/libvir-list
>>>
>>
>> Scott,
>>
>> I've got the exact same CPUs (Dell PowerEdge R815). You'll find the
>> NUMA info is detected node info is detected incorrectly. Oddly we have
>> different data detected. What kernel are you running?
>>
>> with libvirt 1.0.1
>> root host ~ # virsh nodeinfo
>> CPU model:           x86_64
>> CPU(s):              64
>> CPU frequency:       2100 MHz
>> CPU socket(s):       1
>> Core(s) per socket:  64
>> Thread(s) per core:  1
>> NUMA cell(s):        1
>> Memory size:         132013200 KiB
>>
>> virsh capabilities is a bit "smarter" but still not 100%. It sees 8
>> NUMA nodes and puts the CPUs in the correct nodes but still detects
>> the CPU incorrectly.
>>
>>
>
> Doug,
>
> I'm running a RedHat kernel; 2.6.32-220.2.1.el6.x86_64 specifically.
>
> Thanks for the insight of your experiences, very interesting. I find the
> same that the NUMA info appears detected incorrectly for this processor.

Scott,

It appears that the CPU info isn't detected correctly because it can't
be detected correctly or really expressed in the values we know/have.
For some info you can see:
https://bugzilla.redhat.com/show_bug.cgi?id=874050

The issue is that libvirt and to extent the Linux kernel expect 1
physical socket to equate to 1 NUMA node. In the case of these
processors there are really 2 individual "packages" in 1 physical
socket. Each package has its own memory controller and that is tied to
8 cores. Hence how you get 8 NUMA nodes with 8 processors each.
However to add to the confusion AMD has implemented their own type of
hyperthreading in that they have two separate cores but 2 cores share
1 FPU. So in reality each package has 8 cores but if you're doing FPU
work, you only have 4 per package.

The previous decision was that 4 sockets, 8 cores per socket and 2
thread per core was not a correct way to express this CPU since it is
true that each core separate. But the current solution is IMHO worse
by saying 1 socket, 64 cores, 1 thread per core. A better result would
be to say 8 sockets, 8 cores per socket, and 1 thread per core. But
that would require us to break the idea that a socket is a physical
socket and align a socket with a "package".

Anyone else thoughts? Would this be acceptable? Right now the output
is broken and if you put any NUMA values into the XML or use numad the
domain will not start and will give a cryptic error message. Which is
a lot worst than reporting 4 sockets, 8 cores, 1 thread or 8 sockets,
8 cores and 1 thread IMHO.

-- 
Doug Goldstein


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]