Re: How to mirror web sites

[daverio cri ensmp fr (Laurent DAVERIO)]

> > Have you tried GNU Wget (should be version 1.4.x) ? I think you might find
> > it in RPM form in any RedHat mirror site.
> > 
> 
> I have tried this, but I find that sometimes it fails to follow certain of
> the links on a page despite them being at the same site and at the same
> recursion level, and it's not because it is halting due to having downloaded
> too much data.

What kind of links ? Could they have something to do with the robots.txt file ?

Laurent.