[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Memory usage - default X install can't use yum with 1GB RAM machine



On Thu, 2007-10-11 at 15:34 -0400, Paul Wouters wrote:
> Hi,
> 
> This isue is problably known, but I wanted to bring it up anyway.
> 
> ExecSum: Running a 64bit Fedora 7 Test 2 with 1GB RAM fails to update

 Yes, we want yum to use less memory and be faster, hoever...

> Long story: Yum failed me while trying to update from rawhide today:

 Not quite.

> Transaction Summary
> =============================================================================
> Install     50 Package(s)
> Update     702 Package(s)
> Remove       1 Package(s)
> 
> Total download size: 976 M
> Is this ok [y/N]: y
> Downloading Packages:
> [...]
> Transaction Test Succeeded
> Running Transaction
>   Updating  : libgcc                       ################### [   1/1461]
> error: Couldn't fork %post: Cannot allocate memory
> 
> The system was running X with default daemons. Memory usage shows this
> as the top five users:
> 
> Tasks: 137 total,   2 running, 135 sleeping,   0 stopped,   0 zombie
> Cpu(s): 10.6%us,  1.3%sy,  0.0%ni, 88.1%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Mem:   1013676k total,   782344k used,   231332k free,     8788k buffers
> Swap:   522104k total,   452872k used,    69232k free,   164116k cached

 You have 522MB of Swap, and 450MB is used. Add more swap. You may also
find that you have a lot of dead memory due to updated files. prelink
esp. is great for this.
 I've attached a simple python script I've used, it's not perfect but it
shows roughly how much dead memory you have and will kind of show where
it is.

> When yum died, i checked the above usage, so it looks like yum tried to
> use more then 300Mb of non shared RAM. That's pretty insane. Worst thing,
> is that it had *completed* all the dependancy checks, so I'm a bit confused
> why it still needs so much memory. All it needs at that point (it was
> installing the 1st rpm), is the list of 752 package filenames in an
> install-order list. Why is that taking up 300MB?

 300MB to upgrade 1GB of data is excessive?

-- 
James Antill -- <james antill redhat com>
"Please, no.  Let's not pull in a dependency for something as simple as a
string library." -- Kristian Høgsberg <krh redhat com>
#! /usr/bin/python -t


import os


# Decent (UK/US English only) number formatting.
import locale
locale.setlocale(locale.LC_ALL, '') 

def loc_num(x):
    """ Return a string of a number in the readable "locale" format. """
    return locale.format("%d", int(x), True)

pids = []
for d in os.listdir("/proc/"):
    try:
        pid = int(d)
        pids.append(pid)
    except:
        pass

def map_sz(x):
    (beg, end) = x.split('-')
    return (int(end, 16) - int(beg, 16))

files = {}
for pid in pids:
    try:
        try:
            lines = open("/proc/%d/smaps" % pid).readlines()
            smaps = True
        except:
            lines = open("/proc/%d/maps" % pid).readlines()
            smaps = False

        off = 0
        while off < len(lines):
            line = lines[off]
            off += 1
            try:
                int(line[0])
            except:
                continue

            data = line.split(None, 5)
            try:
                ino = int(data[4])
                dev = int(data[3].split(":", 2)[0], 16)
            except:
                print "DBG: Bad line:", lines[off - 1]
                print "DBG:     data=", data
                continue
                
            if dev == 0:
                continue
            if ino == 0:
                continue
            if '(deleted)' not in data[5]:
                continue

            key = "%s:%d" % (data[3], ino)
            if key not in files:
                files[key] = lambda x: x # Hack
                
                files[key].s_size          = 0
                files[key].s_rss           = 0
                files[key].s_shared_clean  = 0
                files[key].s_shared_dirty  = 0
                files[key].s_private_clean = 0
                files[key].s_private_dirty = 0
                files[key].referenced      = 0
                
                files[key].vsz  = 0
                files[key].pids = []
                files[key].name = data[5]
                
            files[key].vsz += map_sz(data[0])
            try:
                if smaps:
                    files[key].s_size          += int(lines[off].split(None, 3)[1])
                    off += 1
                    files[key].s_rss           += int(lines[off].split(None, 3)[1])
                    off += 1
                    files[key].s_shared_clean  += int(lines[off].split(None, 3)[1])
                    off += 1
                    files[key].s_shared_dirty  += int(lines[off].split(None, 3)[1])
                    off += 1
                    files[key].s_private_clean += int(lines[off].split(None, 3)[1])
                    off += 1
                    files[key].s_private_dirty += int(lines[off].split(None, 3)[1])
                    off += 1
                    try:
                        files[key].referenced  += int(lines[off].split(None, 3)[1])
                        off += 1
                    except:
                        pass
            except:
                print "DBG: Bad data:";, lines[off - 1]
                
            files[key].pids.append(pid)
    except:
        pass

vsz             = 0
s_size          = 0
s_rss           = 0
s_shared_clean  = 0
s_shared_dirty  = 0
s_private_clean = 0
s_private_dirty = 0
referenced      = 0

for x in files.values():
    vsz             += x.vsz
    s_size          += x.s_size
    s_rss           += x.s_rss
    s_shared_clean  += x.s_shared_clean
    s_shared_dirty  += x.s_shared_dirty
    s_private_clean += x.s_private_clean
    s_private_dirty += x.s_private_dirty
    referenced      += x.referenced

    print x.pids, loc_num(x.vsz), x.name,
print "\
=============================================================================="
print "num             = %-20s" % loc_num(len(files))
print "vsz             = %-20s" % loc_num(vsz)
print "\
------------------------------------------------------------------------------"
print "s_size          = %-20s" % loc_num(s_size * 1024)
print "s_rss           = %-20s" % loc_num(s_rss * 1024)
print "s_shared_clean  = %-20s" % loc_num(s_shared_clean * 1024)
print "s_shared_dirty  = %-20s" % loc_num(s_shared_dirty * 1024)
print "s_private_clean = %-20s" % loc_num(s_private_clean * 1024)
print "s_private_dirty = %-20s" % loc_num(s_private_dirty * 1024)
print "referenced      = %-20s" % loc_num(referenced * 1024)
print "\
=============================================================================="

Attachment: signature.asc
Description: This is a digitally signed message part


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]