Login | Register
My pages Projects Community openCollabNet

Discussions > dev > Signifigant speedup patch to scanChannel

Project highlights: Stable Version: 1.6.1, Development Version: 1.7.6

current
Discussion topic

2020-03-13: This site is going to be decommissioned and shut down very soon. Please copy and archive any data you wish to keep ASAP

Back to topic list

Signifigant speedup patch to scanChannel

Reply

Author Frank Sweetser <fs at WPI dot EDU>
Full name Frank Sweetser <fs at WPI dot EDU>
Date 2004-01-13 14:38:03 PST
Message Attached is a patch to db/postgres/postgres.py that changes the way that files
are scanned. Rather than doing (roughly)

list = os.listdir(dir)
for file in list:
    if not self._isInDB(file):
        add(file)

it now does

existing = self._listFiles
list = os.listdir(dir)

for file in list:
    if not file in existing:
        add(file)

This means that while it does take a little more memory, it only has to hit
the database twice - once for src and once for bin. On my system, memory
usage by apache is still under 9M, and the time for scanning a full RH 7.3
plus updates with no new packages goes from 5m 15s down to 1m 30s.

For my next trick, I plan to teach scanChannel how to populate a directory
structure for use with yum...

--
Frank Sweetser fs at wpi.edu
WPI Network Engineer


--------------------​--------------------​--------------------​---------
To unsubscribe, e-mail: dev-unsubscribe@curr​ent.tigris.org
For additional commands, e-mail: dev-help at current dot tigris dot org

« Previous message in topic | 1 of 2 | Next message in topic »

Messages

Show all messages in topic

Signifigant speedup patch to scanChannel Frank Sweetser <fs at WPI dot EDU> Frank Sweetser <fs at WPI dot EDU> 2004-01-13 14:38:03 PST
     Re: Signifigant speedup patch to scanChannel Frank Sweetser <fs at WPI dot EDU> Frank Sweetser <fs at WPI dot EDU> 2004-01-14 09:12:09 PST
Messages per page: