[Fedora-livecd-list] [PATCH] --turbo-liveinst improves livecd installer speed by about 20%

Douglas McClendon dmc.fedora at filteredperception.org
Tue Jul 24 23:57:52 UTC 2007


Jeremy Katz wrote:
> On Mon, 2007-07-23 at 03:06 -0500, Douglas McClendon wrote:
>> They implement what I have described rather verbosely on fedora-livecd-list and 
>> in bug 248082.
>>
>> The short story is the fedora7 livecd installer works by copying a 4G ext3 image 
>> to the destination rootfs, and then resizing to maximal size.
>>
>> The attached patches improve the speed of this step by 10-30% (for cdrom vs fast 
>> livecd-iso-to-disk'd usbflash install media respectively).
>>
>> This is accomplished because the 4G image actually only holds 2G of data.  The 
>> patch to livecd-creator, when invoked with --turbo-liveinst, will create a small 
>> (25kb) binary delta file on the livecd.  The patch to anaconda, will detect the 
>> presence of the file, and if it is there, use it with devicemapper to create a 
>> 2G ext3 image, which can naturally be copied to the destination volume more 
>> quickly.  (I.e. 2G of zeros don't get written to disk).
> 
> So I'm still not convinced that the wins here are really substantial
> enough given the additional contortions that we have to go through to
> get things going. 

I understand your attitude.  I will however do another round of polishing and 
performance testing in an attempt to convince you.  Ultimately, I hope to at 
least convince you to do your own side-by-side taste test and confirm my 
results.  :)


  Or that even if the win is worthwhile, that it's
> really the best path to take.  Continuing to layer more levels of
> device-mapper indirection just leads to hitting ... interesting cases
> within the kernel as well as making it a lot harder to tell what's going
> on.  

For the sake of argument (i.e. I'm not fanatically attached to this patch), I 
would say that the interesting kernel cases, are probably useful ones to get the 
bugs out of sooner rather than later.  As for making it harder to tell whats 
going on, I think after another round of polishing, it should actually not be 
terribly complicated.  Actually, after you see some of the patches I'll be 
submitting in the coming months with other device mapper tricks, you might look 
fondly on the days when you thought this was too much complexity :) :) :)

> For avoiding the writes and speeding things up, could we instead not be
> smarter in the copying we use to actually take the sparseness into
> account?  

Definitely a valid and interesting option.  In the same vein, as I alluded to 
elsewhere, I think it might be interesting to see if squashfs could be made to 
support sparse files.  I _think_ it currently knows nothing of sparseness.  And 
the way I think it breaks up things into chunks, I suspect that the amount of 
space the 2G of sparseness in os.img, takes up in the squashed image, is 
significant. (I may test that one of these days and post my findings).

Alas, my specialty is in throwing together commandline tools in devious ways.  I 
could probably become knowledgable about ext3 and squashfs internals, but it's 
not really what I want to do in the near future.  The fact that I can work 
around the absence of cool fs features with device mapper tricks...  I just find 
very amusing, in a brain twisting sort of way.

> 
> As for being able to get to a smaller rootfs, I think that if you really
> want to get to requiring a small root, then we have to entirely change
> the copying to be more of a "copy the bits from the filesystem" as
> opposed to a block-level copy.  That would also avoid the duplicated
> copies if you have, eg, a separate /usr (and also avoid the overhead of
> needing that space on the rootfs).  This would also end up avoiding the
> copying overhead as well as the filesystem resizing, although you'd then
> have to format the entire fs.  I think I'd be slightly more interested
> in doing that, although care has to be taken to ensure that we really
> are preserving everything then.

I agree with all of this.  Still, I go back to the existing patch, which 
hopefully should be a bit more elegant and polished soon, and suggest that it is 
at least worth seriously considering for F8.

> 
>> This patch is less than polished and elegant.  But I'm hoping it might make it 
>> into f8t1.  Please review, and give suggestions for improvements.  I'll plan on 
>> trying to clean it up in whatever ways seem best over the next couple of days.
> 
> I think trying to do this path for test1 is a little optimistic,
> although now we've got a good start to be able to be doing some
> experimentation immediately post-test1.  

agreed.  definitely not enough burn in testing for test1.  But as with 
persistence, if the test1 devfreeze contributed to my motivation to get 
something out the door, all the better.

> 
>> (note, the cleanupDeleted patch I just posted to livecd-tools is included here, 
>> as it lays the foundation for this...  as well as reclaiming lots (5-15+%) of 
>> wasted space on the livecd)
> 
> For future purposes, note that it's a lot easier if you don't do this.
> I mean, interdiff works but it just makes it harder to do a direct
> review.  Saying that it depends on an earlier patch is simple to deal
> with.

Yeah, it was just laziness on my part.  Not entirely related, but I still need 
to learn more about git, and where the heck 'diff --git' comes from.

-dmc




More information about the Fedora-livecd-list mailing list