How to change SMP affinity of an IRQ on Ubuntu PV domU inside Xen XCP?

Solution 1:

I do not think that moving interrupts to different CPUs - especially for handling network events - would increase the performance.

The contrary will happen, since the network-code can not be held in a specific CPU any longer.

So as long as you do not experience dropped packets on your network interface, I would say - this is quite normal behaviour for a network that serves many packets.

You need to lower the number of interrupts - moving them around will not help (on the contrary, as I tried to outline).

I see two possible solutions:

• Raise your MTU-size to lower the number of interrupts
• PIN the vCPU 0 of the DomU to a dedicated CPU on the Dom0 (that is not used by any other VM or the Dom0).

Update 2012-12-17: Since you asked for authoritative links - I tried to ask a general question When not to use virtualisation - I think this is one of the cases, where you are hitting general VM limits. One of the answers to the question contains a different approach: Use containers, instead of virtualization.

I hope this helps...

Solution 2:

There is a file called Documentation/IRQ-affinity.txt in the Linux source code.

/proc/irq/IRQ#/smp_affinity specifies which target CPUs are permitted
for a given IRQ source. It's a bitmask of allowed CPUs. It's not allowed
to turn off all CPUs, and if an IRQ controller does not support IRQ
affinity then the value will not change from the default 0xffffffff.

The catch here is that the bitmask is in hex. So, if you have N CPUs,

N=$(grep -c processor /proc/cpuinfo)

to enable all IRQs for all CPUs, where you have N CPUs you can

printf %x $((2**N-1)) | sudo tee /proc/irq/*/smp_affinity

Kernel 3.0 and later has a file called smp_affinity_list. This file takes a comma separated list of CPUs or CPU ranges. Valid examples: 0, 2,3,5-7, 0-7. The previous command is equivalent to:

echo 0-$((N-1)) | sudo tee /proc/irq/*/smp_affinity_list