[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Proposed F13 feature: drop separate updates repository



On 12/02/2009 05:58 PM, Seth Vidal wrote:
> 
> 
> On Wed, 2 Dec 2009, Peter Jones wrote:
> 
>> (on my on tangent...)
>>
>> On 12/02/2009 12:48 PM, Jesse Keating wrote:
>>> I hypothesize that we could place all rpms for a given release
>>> in a single directory (seth will hate this as he wants to split them up
>>> based on first letter of their name for better filesystem performance),
>>
>> Ugh, first letter isn't really a great plan anyway. First (few) letters
>> of a hash of the filename is much better, but obviously hurts
>> browsability.
>> Next best is probably something like how a dead-tree dictionary index
>> works;
>> list everything, split the list up by starting letters evenly, so the
>> directories (given a really unlikely hypothetical package set) are
>>
>> 0/    # contains packages named 0 through 3*
>> 4/    # 4 through 9*
>> a/    # a through ay*
>> az/    # az through bw*
>> bx/    # bx through cz*
>> da/    # da through whatever's next
>> ...
>>
>> so that every directory has about the same number of things.
> 
> If you're looking for perfect division, sure - but the reality is this:
> 
> 19K items in a single dir and ext3 and nfs and many many other things
> crap themselves returning that list.
> 
> If you make 36 subdirs (26+10) performance gets DRAMATICALLY better for
> producing the same list of files.
> 
> I tested it on our backend to be sure. getting the complete pkglist goes
> from taking 5 minutes to take 30s.
> 
> yes, I said 5 minutes.

Yeah, I buy that.

-- 
        Peter

I'd like to start a religion. That's where the money is.
		-- L. Ron Hubbard to Lloyd Eshbach, in 1949;
			quoted by Eshbach in _Over My Shoulder_.


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]