[Date Prev][Date Next] [Thread Prev][Thread Next] [Thread Index] [Date Index] [Author Index]

Re: FedoraJobs: Calls For Hacks

From: Ivan Leong <lstfedoradevel mw hn org>
To: fedora-devel-list redhat com
Subject: Re: FedoraJobs: Calls For Hacks
Date: Sat, 24 Jan 2004 02:34:31 +0800

hi bill
got here a perl script that might fit the bill :)
Requires: Time::Local perl module and curl (or wget or HTTP:: perl module)

you also didn't specify the environment, ie, how often it runs, against which URL (the thread list <http://www.redhat.com/archives/fedora-announce-list/2004-January/thread.html>?), etc. so i made the script get the monthly thread list, parse for the list of msgs in that month, retrieve each msg in sequence, parse the html for a valid announcement. when all msgs have been parsed, it generates a html for each msg, and one RSS xml and one RSS_SECURITY xml.

let me know how i can further tailor the script to your evnironment.

Bill Nottingham wrote:

It's like http://www.gnome.org/bounties/. Except, simpler. And
without any cash changing hands. (Sorry.)

Basically, there's stuff that could be done. And never enough
time to do it all, obviously. So, why not call for volunteers?

Here's an example to run with...

Thomas Chung has listings of the updates at
 http://fedoranews.org/updates/

This is good, but we'd like to have that on fedora.redhat.com,
of course. So, what we need is the following:

Hack: Update Mangler/Archiver

Take an update announcement, say:

http://www.redhat.com/archives/fedora-announce-list/2004-January/msg00002.html

Write a script (python/perl/shell, python probably preferred) that
takes the announcement, and does the following:

1) spits out into a file some prettified HTML output
2) updates a RSS xml file, with the following format
guid=the update id
link=the html file
description=the 'update information' section
(or the changelog, if 'update information' is blank
3) update a *separate* RSS feed as well, if there's
[SECURITY] in the update ;)

You could even dump the entire advisory into xml and xslt it into
whatever, if you so desire.

Send code in response to this message, fedora-devel-list redhat com,
or to me, <notting redhat com>. Send flames to /dev/null. :)

Bill

#!/usr/bin/perl

use strict;
use warnings;
use Time::Local;

# convert to rss (rfc822) dateformat
sub rssdate {
  my @d=split '-',shift;
  my $z;
  my @ww=("Sun","Mon","Tue","Wed","Thu","Fri","Sat");
  my @mth=("Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct",
           "Nov","Dec");
  my ($mday,$mon,$year,$wday);

  ($z,$z,$z,$mday,$mon,$year,$wday,$z,$z)=localtime(
      timelocal(0,0,0,$d[2],$d[1]-1,$d[0]));
  return sprintf "%s, %02d %s %d 00:00:00 GMT",$ww[$wday],$mday,
      $mth[$mon],$year+1900;
}

# output rss
sub outrss {
  my $fname=shift;
  my $data=shift;
  my $sec=shift;
  my $id;

  open OUT,">$fname";
  print OUT "<rss version=\"2.0\">\n".
            "<channel>\n".
            "  <title></title>\n".
            "  <link>link</link>\n".
            "  <description></description>\n".
            "  <item>\n";
  foreach $id (sort {$$data{$b}{date} cmp $$data{$a}{date}} keys %$data) {
    if (($$data{$id}{sec}|$sec)==1) {
      print OUT "    <guid>$id</guid>\n".
                "    <pubDate>".(rssdate($$data{$id}{date}))."</pubDate>\n".
                "    <description>$$data{$id}{desc}</description>\n";
    }
  }
  print OUT "  </item>\n".
            "</channel>\n".
            "</rss>\n";
  close OUT;
}

# output html
sub outhtml {
  my $data=shift;
  my $id;

  foreach $id (keys %$data) {
    open OUT,">$id.html";
    print OUT "<html>\n".
              "<head>\n".
              "<title></title>\n".
              "</head>\n".
              "<body>\n";
    print OUT "<table>\n".
              "<tr><td>Name</td><td>$$data{$id}{name}</td></tr>\n".
              "<tr><td>Version</td><td>$$data{$id}{ver}</td></tr>\n".
              "<tr><td>Release</td><td>$$data{$id}{rel}</td></tr>\n".
              "<tr><td>Summary</td><td>$$data{$id}{sum}</td></tr>\n".
              "<tr><td>Description</td><td>$$data{$id}{desc}</td></tr>\n".
              "</table><hr>\n";
    print OUT "<h4>Update Information</h4>\n<p>$$data{$id}{upd}</p><hr>\n"
        if (defined($$data{$id}{upd}));
    print OUT "<h4>Changelog</h4>\n$$data{$id}{chg}<hr>\n"
        if (defined($$data{$id}{chg}));
    print OUT "</body>\n".
              "</html>\n";
    close OUT;
  }
}

# process a html page for fedora update annoucement
# return 0 for no error, 1 not-an-annoucement, 2 error
my @errstr=("OK","Not an annoucement","Error!");
sub process {
  my $url=shift;
  my $upd=shift;
  my ($html,%upd,$id,$sec);

  $html="";
  open IN,"curl -s \"$url\"|";
  while (<IN>) {
    $html.=$_;
  }
  close IN;

  # remove DOS-styled CRLF
  $html=~s/\cm+//g;

  #
  # do processing
  ####
  my $de="-"x69;
  return 1 if ($html!~/Fedora\s(?:Security\s)?Update\sNotification/);

  if ($html=~/<em>subject<\/em>: (.+?)<\/li>/is) {
    $sec=($1=~/\[SECURITY\]/)?1:0;
  } else {
    print "ERR: parsing HTML: cannot find subject\n";
    return 2;
  }

  $html=$1 if ($html=~/<pre>(.+)<\/pre>/is);

  if ($html!~/----\n
              Fedora\s(?:Security\s)?Update\sNotification\n
             (FEDORA-\d+-\d+)\n
             (\d{4}-\d{2}-\d{2})\n
              ----/x
  ) {
    print "ERR: parsing HTML: cannot find id\n";
    return 2;
  }
  $html=$'; $id=$1; $$upd{$id}{date}=$2; $$upd{$id}{sec}=$sec;

  if ($html!~/\nName\s+:\s+(.+?)\n
                Version\s+:\s+(.+?)\n
                Release\s+:\s+(.+?)\n
                Summary\s+:\s+(.+?)\n
                Description\s:\n/x) {
    print "ERR: parsing HTML: cannot find name\n";
    return 2;
  }
  $html=$';
  $$upd{$id}{name}=$1; $$upd{$id}{ver}=$2; $$upd{$id}{rel}=$3; $$upd{$id}{sum}=$4;
  $$upd{$id}{name}=~s/[ \t]*$//mg;
  $$upd{$id}{ver}=~s/[ \t]*$//mg;
  $$upd{$id}{rel}=~s/[ \t]*$//mg;
  $$upd{$id}{sum}=~s/[ \t]*$//mg;

  if ($html!~/(.+?)\n\n$de/s) {
    print "ERR: parsing HTML: cannot find description\n";
    return 2;
  }
  $html=$'; $$upd{$id}{desc}=$1;
  $$upd{$id}{desc}=~s/([^- ])\n/$1 /g;
  $$upd{$id}{desc}=~s/^[ \t\n]*//g;
  $$upd{$id}{desc}=~s/[ \t\n]*$//g;


  if ($html!~/(?:Update Information:)?
              (?:\s+(.+?)$de)?
              (?:\s+(\*\s(?:Sun|Mon|Tue|Wed|Thu|Fri|Sat).+?)$de)?
                 \s+This\supdate\scan\sbe\sdownloaded\sfrom/sx) {
    print "ERR: parsing HTML: cannot find update\n";
    return 2;
  }
  $html=$';
  my $tmp1=$1; my $tmp2=$2;
  if (defined($tmp1)) {
    if ($tmp1=~/^\*\s(?:Sun|Mon|Tue|Wed|Thu|Fri|Sat)/) {
      $$upd{$id}{chg}=$tmp1;
    } else {
      $$upd{$id}{upd}=$tmp1;
      $$upd{$id}{chg}=$tmp2 if (defined($tmp2));
    }
  }
  if (defined($$upd{$id}{chg})) {
    $$upd{$id}{chg}=~s/^[ \t\n]*//g;
    $$upd{$id}{chg}=~s/[ \t\n]*$//g;
  }
  if (defined($$upd{$id}{upd})) {
    $$upd{$id}{upd}=~s/([^- ])\n/$1 /g;
    $$upd{$id}{upd}=~s/^[ \t\n]*//g;
    $$upd{$id}{upd}=~s/[ \t\n]*$//g;
  }

  return 0;
}

my ($rc,%upd);
my $html="";
my $urlbase="http://www.redhat.com/archives/fedora-announce-list/2004-January";;
open IN,"curl -s $urlbase/thread.html|";
while (<IN>) {
  chomp;
  $html.=$_;
}
close IN;
 
while ($html=~/<a[^>]+?href=(['"]?)(msg\d+\.html)\1[^>]*>/) {
  $html=$';
  $rc=process("$urlbase/$2",\%upd);
  print "$2 ProcessStatus $errstr[$rc]\n";
}

#
# do output
####
outrss("rss.xml",\%upd,1);
outrss("rss_sec.xml",\%upd,0);
outhtml(\%upd);

Name	kernel
Version	2.4.22
Release	1.2149.nptl
Summary	The Linux kernel (the core of the Linux operating system)
Description	The kernel package contains the Linux kernel (vmlinuz), the core of your Fedora Core Linux operating system. The kernel handles the basic functions of the operating system: memory allocation, process allocation, device input and output, etc.

Changelog

* Wed Jan 07 2004 Dave Jones <davej redhat com> - Merge several EXT2/3 fixes from 2.4.25pre - EXT2/3 fixes. - Reclaim pages in truncate - 2.6 EA symlink compatibility - forward-compatibility: online resizing - Allow filesystems with expanded inodes to be mounted - Handle j_commit_interval == 0 - IDE timeout race fix - Merge some 2.4.23pre patches that were missed. - Make root a special case for per-user process limits. - out_of_memory() locking - Drop module count if lockd reclaimer thread failed to start - Fix potential fsync() race condition - s/Red Hat/Fedora/ in specfile (#112992) - Add PCI ident for new Intel e1000 card. (#105046) - Actually wire up 3c59x ethtool ioctl. - Fix up numeric sysctls to match mainline.

Name	httpd
Version	2.0.48
Release	1.2
Summary	Apache HTTP Server
Description	Apache is a powerful, full-featured, efficient, and freely-available Web server. Apache is also the most popular Web server on the Internet.

Update Information

Update Information: This update includes the latest stable release of Apache httpd 2.0, including a fix for the security issue CVE CAN-2003-0542, a buffer overflow in the parsing of configuration files.

Changelog

* Wed Nov 19 2003 Joe Orton <jorton redhat com> 2.0.48-1.2 - bug fix for #110184 * Tue Oct 28 2003 Joe Orton <jorton redhat com> 2.0.48-1.1 - update to 2.0.48 (#108608, thanks to Robert Scheck) - includes security fix for CVE CAN-2003-0542 - reinstate mpm_common.h (#108080)

Name	php
Version	4.3.4
Release	1.1
Summary	The PHP HTML-embedded scripting language. (PHP: Hypertext Preprocessor)
Description	PHP is an HTML-embedded scripting language. PHP attempts to make it easy for developers to write dynamically generated webpages. PHP also offers built-in database integration for several commercial and non-commercial database management systems, so writing a database-enabled webpage with PHP is fairly simple. The most common use of PHP coding is probably as a replacement for CGI scripts. The mod_php module enables the Apache Web server to understand and process the embedded PHP language in Web pages.

Update Information

Update Information: This update includes the latest stable release of PHP 4 with a large number of bug fixes since the previous 4.3.3 release.

Changelog

* Mon Nov 10 2003 Joe Orton <jorton redhat com> 4.3.4-1.1 - rebuild for FC1 updates * Mon Nov 10 2003 Joe Orton <jorton redhat com> 4.3.4-1 - update to 4.3.4 - include all licence files - libxmlrpc fixes

Name	glibc
Version	2.3.2
Release	101.4
Summary	The GNU libc libraries.
Description	The glibc package contains standard libraries which are used by multiple programs on the system. In order to save disk space and memory, as well as to make upgrading easier, common system code is kept in one place and shared between programs. This particular package contains the most important sets of shared libraries: the standard C library and the standard math library. Without these two libraries, a Linux system will not function.

Update Information

Update Information: This glibc update fixes lots of bugs in the regular _expression_ matcher and speeds it up. It fixes a couple of other bugs as well.

Changelog

* Tue Jan 06 2004 Jakub Jelinek <jakub redhat com> 2.3.2-101.4 - some further regex speedups - fix re.translate handling in regex (#112869) - change regfree to match old regex behaviour (what is freed and clearing of freed pointers) * Tue Dec 30 2003 Jakub Jelinek <jakub redhat com> 2.3.2-101.3 - fix pmap_set fd and memory leak (#112726) - fix backreference handling in regex * Tue Dec 30 2003 Jakub Jelinek <jakub redhat com> 2.3.2-101.2 - fix to make pthread_setcancelstate (PTHREAD_CANCEL_DISABLE, ) really disable cancellation (#112512) - lots of regex fixes and speedups (#110401) - fix nextafter*/nexttoward* - handle 6th syscall(3) argument on AMD64 - handle memalign/posix_memalign in mtrace - fix linuxthreads memory leak (#112208) - remove throw () from cancellation points in linuxthreads (#112602) - fix NPTL unregister_atfork - fix unwinding through alternate signal stacks - fix atan2 - fix pshared condvars in NPTL - fix pthread_attr_destroy for attributes created with pthread_attr_init GLIBC_2 0 - add BuildPrereq texinfo (#110252) - fix ceill/floorl on AMD64 - work around IA64 gas bug with unwind info and .align - fix NPTL configure - allow dlopen after fork () in threaded programs - compute IA-64 default thread stack size correctly - fix thread stacks with ulimit -s not a multiple of a page size - randomize PIE shared libraries, honor LD_USE_LOAD_BIAS env variable - fix execstack handling on kernels without exec-shield

Name	kernel
Version	2.4.22
Release	1.2138.nptl
Summary	The Linux kernel (the core of the Linux operating system)
Description	The kernel package contains the Linux kernel (vmlinuz), the core of your Red Hat Linux operating system. The kernel handles the basic functions of the operating system: memory allocation, process allocation, device input and output, etc. Paul Starzetz discovered a flaw in bounds checking in mremap() in the Linux kernel versions 2.4.23 and previous which may allow a local attacker to gain root privileges. No exploit is currently available; however, it is believed that this issue is exploitable (although not trivially.) The Common Vulnerabilities and Exposures project (cve.mitre.org) has assigned the name CAN-2003-0985 to this issue. All users are advised to upgrade to these errata packages, which contain a backported security patch that corrects this issue. Red Hat would like to thank Paul Starzetz from ISEC for disclosing this issue as well as Andrea Arcangeli and Solar Designer for working on the patch. These packages also contain a fix for a minor information leak in the real time clock (rtc) routines. The Common Vulnerabilities and Exposures project (cve.mitre.org) has assigned the name CAN-2003-0984 to this issue.

Changelog

* Wed Dec 24 2003 Dave Jones <davej redhat com> - Fix mremap corner case. * Tue Dec 23 2003 Dave Jones <davej redhat com> - Numerous USB fixes (#110307, #90442, #107929, #110872) * Tue Dec 16 2003 Dave Jones <davej redhat com> - Fix leak in CDROM IOCTL. (#112249)

Name	kernel
Version	2.4.22
Release	1.2140.nptl
Summary	The Linux kernel (the core of the Linux operating system)
Description	The kernel package contains the Linux kernel (vmlinuz), the core of your Red Hat Linux operating system. The kernel handles the basic functions of the operating system: memory allocation, process allocation, device input and output, etc.

Update Information

Various RTC drivers had the potential to leak small amounts of kernel memory to userspace through IOCTL's. The Common Vulnerabilities and Exposures project (cve.mitre.org) has assigned the name CAN-2003-0984 to this issue.

<rss version="2.0">
<channel>
  <title></title>
  <link>link</link>
  <description></description>
  <item>
    <guid>FEDORA-2003-004</guid>
    <pubDate>Thu, 08 Jan 2004 00:00:00 GMT</pubDate>
    <description>Apache is a powerful, full-featured, efficient, and freely-available Web server. Apache is also the most popular Web server on the Internet.</description>
    <guid>FEDORA-2003-047</guid>
    <pubDate>Wed, 07 Jan 2004 00:00:00 GMT</pubDate>
    <description>The kernel package contains the Linux kernel (vmlinuz), the core of your Red Hat Linux operating system.  The kernel handles the basic functions of the operating system:  memory allocation, process allocation, device input and output, etc.</description>
    <guid>FEDORA-2003-046</guid>
    <pubDate>Mon, 05 Jan 2004 00:00:00 GMT</pubDate>
    <description>The kernel package contains the Linux kernel (vmlinuz), the core of your Red Hat Linux operating system.  The kernel handles the basic functions of the operating system:  memory allocation, process allocation, device input and output, etc. 
Paul Starzetz discovered a flaw in bounds checking in mremap() in the Linux kernel versions 2.4.23 and previous which may allow a local attacker to gain root privileges.  No exploit is currently available; however, it is believed that this issue is exploitable (although not trivially.) The Common Vulnerabilities and Exposures project (cve.mitre.org) has assigned the name CAN-2003-0985 to this issue. 
All users are advised to upgrade to these errata packages, which contain a backported security patch that corrects this issue.   
Red Hat would like to thank Paul Starzetz from ISEC for disclosing this issue as well as Andrea Arcangeli and Solar Designer for working on the patch. 
These packages also contain a fix for a minor information leak in the real time clock (rtc) routines. The Common Vulnerabilities and Exposures project (cve.mitre.org) has assigned the name CAN-2003-0984 to this issue.</description>
  </item>
</channel>
</rss>

<rss version="2.0">
<channel>
  <title></title>
  <link>link</link>
  <description></description>
  <item>
    <guid>FEDORA-2003-048</guid>
    <pubDate>Tue, 13 Jan 2004 00:00:00 GMT</pubDate>
    <description>The kernel package contains the Linux kernel (vmlinuz), the core of your Fedora Core Linux operating system.  The kernel handles the basic functions of the operating system:  memory allocation, process allocation, device input and output, etc.</description>
    <guid>FEDORA-2003-045</guid>
    <pubDate>Mon, 12 Jan 2004 00:00:00 GMT</pubDate>
    <description>The glibc package contains standard libraries which are used by multiple programs on the system. In order to save disk space and memory, as well as to make upgrading easier, common system code is kept in one place and shared between programs. This particular package contains the most important sets of shared libraries: the standard C library and the standard math library. Without these two libraries, a Linux system will not function.</description>
    <guid>FEDORA-2003-005</guid>
    <pubDate>Thu, 08 Jan 2004 00:00:00 GMT</pubDate>
    <description>PHP is an HTML-embedded scripting language. PHP attempts to make it easy for developers to write dynamically generated webpages. PHP also offers built-in database integration for several commercial and non-commercial database management systems, so writing a database-enabled webpage with PHP is fairly simple. The most common use of PHP coding is probably as a replacement for CGI scripts. The mod_php module enables the Apache Web server to understand and process the embedded PHP language in Web pages.</description>
    <guid>FEDORA-2003-004</guid>
    <pubDate>Thu, 08 Jan 2004 00:00:00 GMT</pubDate>
    <description>Apache is a powerful, full-featured, efficient, and freely-available Web server. Apache is also the most popular Web server on the Internet.</description>
    <guid>FEDORA-2003-047</guid>
    <pubDate>Wed, 07 Jan 2004 00:00:00 GMT</pubDate>
    <description>The kernel package contains the Linux kernel (vmlinuz), the core of your Red Hat Linux operating system.  The kernel handles the basic functions of the operating system:  memory allocation, process allocation, device input and output, etc.</description>
    <guid>FEDORA-2003-046</guid>
    <pubDate>Mon, 05 Jan 2004 00:00:00 GMT</pubDate>
    <description>The kernel package contains the Linux kernel (vmlinuz), the core of your Red Hat Linux operating system.  The kernel handles the basic functions of the operating system:  memory allocation, process allocation, device input and output, etc. 
Paul Starzetz discovered a flaw in bounds checking in mremap() in the Linux kernel versions 2.4.23 and previous which may allow a local attacker to gain root privileges.  No exploit is currently available; however, it is believed that this issue is exploitable (although not trivially.) The Common Vulnerabilities and Exposures project (cve.mitre.org) has assigned the name CAN-2003-0985 to this issue. 
All users are advised to upgrade to these errata packages, which contain a backported security patch that corrects this issue.   
Red Hat would like to thank Paul Starzetz from ISEC for disclosing this issue as well as Andrea Arcangeli and Solar Designer for working on the patch. 
These packages also contain a fix for a minor information leak in the real time clock (rtc) routines. The Common Vulnerabilities and Exposures project (cve.mitre.org) has assigned the name CAN-2003-0984 to this issue.</description>
  </item>
</channel>
</rss>

Follow-Ups:
- Re: FedoraJobs: Calls For Hacks
  - From: Bill Nottingham

References:
- FedoraJobs: Calls For Hacks
  - From: Bill Nottingham

[Date Prev][Date Next] [Thread Prev][Thread Next] [Thread Index] [Date Index] [Author Index]