putting fedora on a (memory) diet... some suggestions for the kernel config

[Don Dutile]

Arjan van de Ven wrote:
> Looking at kernel memory use (based on 2.6.19 with a fedora config),
> there's some not that hard ways to gain quite a bit of memory back.
> 
> Most important is CONFIG_NR_CPUS, right now that is set to 255 for
> everyone; this value is used for scaling a LOT of things in the kernel,
> and the sad thing is it's not even possible today to get a 255
> core/processor machine (you run out of apic ids well before you get that
> far).
> 
> Setting CONFIG_NR_CPUS to 64 (which is still huge, even 16 would be more
> than 99.99999% of the people who use fedora will ever use) will save
> 
The proper solution: kmalloc-based, per-cpu structs; wire to gs(or is it cs) reg
like pda is, if performance is an issue.  then you only get what you need.

> 
> user 		255		64	saving
> ----------------------------------------------
> irq_desc	2154496		294912	1859584
> irq_domain	269312		18432	250880
> irq_lists	134656		36864	97792
> irq_2_pin	100992		27648	73344
> irq_timer_state	67328		18432	48896
> msi_desc	67328		18432	48896
> per_cpu__kstat	33728		9280	24448
> -----------------------------------------------
> total					2403839
> 
> this alone saves 2.4Mb of memory in *static* buffers! Add a bunch more
> in smaller buffers and all dynamic kernel allocations and it'll be
> closer to 3Mb.. Easy gain right there.
> 
> Another one is __log_buf; this is currently 128Kb. Arguably it's
> important for debugging to get 128Kb of dmesg info, but I wonder if 64Kb
> would be enough as well, it's another easy save by setting
> CONFIG_LOG_BUF_SHIFT to 16.
Pls! Size it based on memory: on small mem footprints (256MB), make it 32K;
large memory footprints (1G and above) make it 512KB! (a la "I like to make
it big for debugging" Jeremy).

> 
> A third one is disabling CPU hotplug. CPU hotplug keeps quite a bit of
> code in the kernel, that otherwise is boot-time only and gets evicted
> right after boot..... You could argue the same for PCI hotplug but
> cardbus is sort of the party spoiler there I suppose.
> 
ditto Jeremy's reply wrt multicore/multi-socket systems (they will be std
PC within a year).

> 
> I don't know what Dave will pick for CONFIG_STACK_PROTECTOR_ALL; I would
> suggest to turn this off (while enabling the normal
> CONFIG_STACK_PROTECTOR); _ALL doesn't add much if any security while it
> creates bigger and slower code everywhere. (The difference is that _ALL
> forces the canary on every function, while the normal setting only
> forces it for functions with buffers on the stack)
Another good project: stack utilization reduction.
better yet: put tools in place that warn when a function uses X-amt of stack,
            and analyze those functions/modules for lower stack utilization.

> 
> 
> 
>