Login | Register
My pages Projects Community openCollabNet

Discussions > dev > Signifigant speedup patch to scanChannel

Project highlights: Stable Version: 1.6.1, Development Version: 1.7.6

current
Discussion topic

Back to topic list

Signifigant speedup patch to scanChannel

Reply

Author Frank Sweetser <fs at WPI dot EDU>
Full name Frank Sweetser <fs at WPI dot EDU>
Date 2004-01-13 14:38:03 PST
Message Attached is a patch to db/postgres/postgres.py that changes the way that files
are scanned. Rather than doing (roughly)

list = os.listdir(dir)
for file in list:
    if not self._isInDB(file):
        add(file)

it now does

existing = self._listFiles
list = os.listdir(dir)

for file in list:
    if not file in existing:
        add(file)

This means that while it does take a little more memory, it only has to hit
the database twice - once for src and once for bin. On my system, memory
usage by apache is still under 9M, and the time for scanning a full RH 7.3
plus updates with no new packages goes from 5m 15s down to 1m 30s.

For my next trick, I plan to teach scanChannel how to populate a directory
structure for use with yum...

--
Frank Sweetser fs at wpi.edu
WPI Network Engineer


--------------------​--------------------​--------------------​---------
To unsubscribe, e-mail: dev-unsubscribe@curr​ent.tigris.org
For additional commands, e-mail: dev-help at current dot tigris dot org

« Previous message in topic | 1 of 2 | Next message in topic »

Messages

Show all messages in topic

Signifigant speedup patch to scanChannel Frank Sweetser <fs at WPI dot EDU> Frank Sweetser <fs at WPI dot EDU> 2004-01-13 14:38:03 PST
     Re: Signifigant speedup patch to scanChannel Frank Sweetser <fs at WPI dot EDU> Frank Sweetser <fs at WPI dot EDU> 2004-01-14 09:12:09 PST
Messages per page: