offlist, Re: [Fedora-livecd-list] persistence testing howto

Mike Dickson mdickson at redhat.com
Mon Jan 7 04:59:33 UTC 2008


I used the yum update to simulate a developer install of substantial
size - nothing special about it.  I wanted to see if you wrote a lot to
it what would happen.  I gather we are running into different mount
points filling up?  (Remember I am a developer not a linux guy.)  There
is room on the stick just not in tmp it filled up?

I have a feeling that I will need to spin my own .iso from the Developer
Live CD that includes RHDS and pretty much everything they need.  Then
let them save a few documents in home to keep the large amount of
writing to a minimum.  Am I heading in the right direction?

MikeD

On Fri, 2008-01-04 at 22:46 -0600, Douglas McClendon wrote:

> Mike Dickson wrote:
> > I am out of the chess game.   Any news?
> 
> I'm not sure I follow your analogy?
> 
> Do you understand how the persistence is achieved?  I.e. a devicemapper
> shapshot, where any changed blocks on the root filesystem get written to
> the persistence file.  If the same block gets changed more than once,
> that block gets updated in the persistence file, and thus no more space
> is taken.
> 
> As a result of this, if you do something like a yum update, that creates
> and deletes a bunch of files on the rootfs (in addition to the ones it 
> finally installs and leaves as is), all those changed blocks eat up 
> space in the persistence file, and don't get freed or even reused, 
> unless and until the filesystem decides to write to the exact same block.
> 
> With this lack of ideal efficiency, the question then becomes- is this 
> sufficient for your goals?  I can imagine many usage scenarios in which 
> this is sufficient, and as mentioned in other mails, many ways in which 
> to try and mitigate the ineficiency.
> 
> One thing I'll try when I find the time, is something like doing a
> 
> mkdir /dev/shm/tmpspace
> mkdir /dev/shm/tmpspace/vtmp
> mkdir /dev/shm/tmpspace/tmp
> mkdir /dev/shm/tmpspace/fedora
> mkdir /dev/shm/tmpspace/updates
> mount --bind /dev/shm/tmpspace/tmp /tmp
> mount --bind /dev/shm/tmpspace/vtmp /var/tmp
> mount --bind /dev/shm/tmpspace/fedora /var/cache/yum/fedora/packages
> mount --bind /dev/shm/tmpspace/updates /var/cache/yum/updates/packages
> 
> before I do a yum install of some small package, and then seeing what 
> the difference is in blocks used on the persistence file.
> 
> Anyway, beyond that, I do intend to re-add optional unionfs support to 
> my VirOS livecd creation toolset, despite the fact that it breaks my 
> rebootless installation mechanism.  Hopefully that will be done soon, 
> but probably not for months as I have several other higher priorities at 
> the moment.
> 
> But to be clear, because of all the above, trying to do a yum update 
> even with a 1G persistence file and the above method, is probably not 
> really feasible (except maybe the first day or two after a new release).
> Yum updating a single package to get some specific critical bugfix, now 
> that might be doable.
> 
> The main usage scenario I foresee for the feature, is adding users to 
> the system, a seperate /home in fstab (mounted from a different fsimage 
> file on the same usbstick), and editing configuration files 
> (/etc/dovecot.conf, /etc/sysconfig/* /etc/rc.d/rc.local, etc....)
> And installing a small number of other packages.
> 
> That certainly isn't as nice as if you could do a yum update, and end up 
> only using the same amount of space on the liveusb as if you respun the 
> livecd with the same updates.  But if you figure out a way to do that, I 
> will give you mad props :)
> 
> -dmc
> 
> 
> 
> > 
> > MikeD
> > 
> > On Fri, 2008-01-04 at 02:33 +0000, Mike Dickson wrote:
> >> I just finished trying to download JBoss Developer Studio and 
> >> installing it on the thumb drive.  It filled up again.  I then dropped 
> >> the .jar on the stick hoping that I could install from that and it 
> >> filled up again.  Checkmate. 
> >>
> >> MikeD
> >>
> >> On Thu, 2008-01-03 at 15:30 -0800, Mike Dickson wrote:
> >>> Ran that and yes the snapshot area filled up BEFORE the errors.  Let 
> >>> me know what I can do....
> >>>
> >>> MikeD
> >>>
> >>> On Thu, 2008-01-03 at 16:31 -0600, Douglas McClendon wrote:
> >>>> Mike Dickson wrote:
> >>>> > Guys,
> >>>> > 
> >>>> > I got a LiveCD + Persistence usb drive running from your scripts, but 
> >>>> > got I/O errors if I tried to do a yum update.
> >>>> > 
> >>>> > Before that I was able to vi test.txt and put some text in and it 
> >>>> > survived a reboot.
> >>>> > 
> >>>> > What can I do to address the i/o errors?
> >>>>
> >>>> My first question/explanation would be that you filled up the snapshot 
> >>>> device.  This is quite possible, as a yum install involves creating 
> >>>> several copies of the actual files you end up installing.
> >>>>
> >>>> The way to see if this is what is happening would be to have another 
> >>>> terminal open, and periodically watch the output of "dmsetup status". 
> >>>> As new blocks are written to the rootfs snapshot device, you will see 
> >>>> the snapshot filling up.
> >>>>
> >>>> If you get these IO errors even before the snapshot fills up, please try 
> >>>> to post some more detailed output.
> >>>>
> >>>> In general, as discussed there are pros and cons with this method, and a 
> >>>> unionfs method.  I do think there are ways to work around the cons of 
> >>>> this method in such a way that it is useful.  For instance, I'll play 
> >>>> around and see if I can prescribe a process of using yum that will get 
> >>>> it to create all of its intermediate files in a native tmpfs (/dev/shm 
> >>>> or the like) instead of the rootfs, so that they don't eat into the 
> >>>> snapshot space.  Likewise, now that I have my first actual tester, maybe 
> >>>> I'll figure out some other creative ways to improve the method (I have 
> >>>> some ideas I need to experiment with...).
> >>>>
> >>>> Thanks,
> >>>>
> >>>> -dmc
> >>>>
> >>>>
> >>>>
> >>>> > 
> >>>> > MikeD
> >>>> > 
> >>>> > "Messsage from syslogd at localhost <mailto:syslogd at localhost> at
> >>>> >   kernel: journal commit i/o error"
> >>>> > 
> >>>> > 
> >>>> > On Wed, 2008-01-02 at 04:07 -0800, Mike Dickson wrote:
> >>>> >> I have some time now.  I am attempting this tonight and tomorrow.  I 
> >>>> >> will let you know.
> >>>> >>
> >>>> >> MikeD
> >>>> >>
> >>>> > 
> >>>> > 
> >>>> > ------------------------------------------------------------------------
> >>>> > 
> >>>> > --
> >>>> > Fedora-livecd-list mailing list
> >>>> > Fedora-livecd-list at redhat.com <mailto:Fedora-livecd-list at redhat.com>
> >>>> > https://www.redhat.com/mailman/listinfo/fedora-livecd-list
> >>>>
> >>> --
> >>> Fedora-livecd-list mailing list
> >>> Fedora-livecd-list at redhat.com <mailto:Fedora-livecd-list at redhat.com>
> >>> https://www.redhat.com/mailman/listinfo/fedora-livecd-list
> >> --
> >> Fedora-livecd-list mailing list
> >> Fedora-livecd-list at redhat.com <mailto:Fedora-livecd-list at redhat.com>
> >> https://www.redhat.com/mailman/listinfo/fedora-livecd-list
> > 
> > ------------------------------------------------------------------------
> > 
> > --
> > Fedora-livecd-list mailing list
> > Fedora-livecd-list at redhat.com
> > https://www.redhat.com/mailman/listinfo/fedora-livecd-list
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/fedora-livecd-list/attachments/20080107/03102593/attachment.htm>


More information about the Fedora-livecd-list mailing list