Login | Register
My pages Projects Community openCollabNet

Discussions > dev > Patch against 1.7.4

Project highlights: Stable Version: 1.6.1, Development Version: 1.7.6

current
Discussion topic

Hide all messages in topic

All messages in topic

Re: Patch against 1.7.4

Reply

Author theslack
Full name Jack Neely
Date 2006-08-11 10:33:02 PDT
Message And committed.

Jack

Re: Patch against 1.7.4

Reply

Author Pauline Middelink <middelink at polyware dot nl>
Full name Pauline Middelink <middelink at polyware dot nl>
Date 2006-08-11 10:20:03 PDT
Message On Fri, 11 Aug 2006 around 19:18:15 +0200, Pauline Middelink wrote:
> On Fri, 11 Aug 2006 around 09:31:08 -0400, Jack Neely wrote:
> > Pauline,
> >
> > Umm...I think you forgot to include the users.py file. :-)
>
> Shoot. I have attached a new diff, straight against svn-257.

Okay, and now its really attached.

    Met vriendelijke groet,
        Pauline Middelink
--
GPG Key fingerprint = 2D5B 87A7 DDA6 0378 5DEA BD3B 9A50 B416 E2D0 C3C2
For more details look at my website http://www.polyware.​nl/~middelink
Attachments

Re: Patch against 1.7.4

Reply

Author Pauline Middelink <middelink at polyware dot nl>
Full name Pauline Middelink <middelink at polyware dot nl>
Date 2006-08-11 10:18:15 PDT
Message On Fri, 11 Aug 2006 around 09:31:08 -0400, Jack Neely wrote:
> Pauline,
>
> Umm...I think you forgot to include the users.py file. :-)

Shoot. I have attached a new diff, straight against svn-257.

PS. It includes all the earlier mentioned fixes.

    Met vriendelijke groet,
        Pauline Middelink
--
GPG Key fingerprint = 2D5B 87A7 DDA6 0378 5DEA BD3B 9A50 B416 E2D0 C3C2
For more details look at my website http://www.polyware.​nl/~middelink
Attachments

Re: Patch against 1.7.4 - update

Reply

Author Pauline Middelink <middelink at polyware dot nl>
Full name Pauline Middelink <middelink at polyware dot nl>
Date 2006-08-11 09:43:42 PDT
Message On Fri, 11 Aug 2006 around 09:31:08 -0400, Jack Neely wrote:

As promised the patch. It fixeds two things,
One correction on the exception stuff, and one for the
compatiblechannels stuff.

    Met vriendelijke groet,
        Pauline Middelink
--
GPG Key fingerprint = 2D5B 87A7 DDA6 0378 5DEA BD3B 9A50 B416 E2D0 C3C2
For more details look at my website http://www.polyware.​nl/~middelink
Attachments

Re: Patch against 1.7.4

Reply

Author Pauline Middelink <middelink at polyware dot nl>
Full name Pauline Middelink <middelink at polyware dot nl>
Date 2006-08-11 09:14:10 PDT
Message On Fri, 11 Aug 2006 around 09:20:45 -0400, Jack Neely wrote:
> Pauline,
>
> Abosultely awesome. Thanks a bunch. I've actually been struggling
> with how to handle user authentication for a while. You'd think this
> is common and there would be libraries or best practices or something
> similarly googlable.

Nah... :)

> Let's see...if you handle an exception in most cases you also want to
> call logException() in the except clause so that the exception get in
> the log file.

It's not a bad exception, its a way to determine how many fields in
the list of tuples is offered.

> I'm not sure I want to implenent random creation of new users via
> up2date's registration thing. But that's very minor and the code is
> definitely needed.

Creation of users is not a problem for me. In my system I want
users to have a list of subscribed channels, to which they can
connect/subscribe their systems. By not giving users channels,
their use for the system is small...

> One of the bigger things I'm aiming for is to
> bring what RHN should be with the up2date client to Yum. (Especially
> since up2date is going away.) In the long run that probably means a
> replacement for rhn_check. But we might be able to get some code from
> Red Hat's RHN Yum plugin when that is available.

If its really going away, than yes. But that means RHN will have a
problem too, since they reporting and stuff is totally baed on up2date

> Hmmm...I probably haven't explained well to my user/devel community my
> "vision" (*chuckle*) for supporting Yum.

:) Nod, nod...

> Your database foo may be stronger than mine. Does it make since to
> have an unknown packages table and make INSTALLED a mapping between
> profile_ids and package_id or unknownpkg_id?

Not sure it is worth the effort. We simply don't have more information
than nvre, and I want to try to keep things clean. It already takes
quite (relative) some time to maintain the package_ids with all INSTALLED
lists.

F.Y.I. I have about 105 systems running each with about 1400 packets
installed. Whoops. 150.000 lines in INSTALLED. :)

> >Something you might want to look at (python is not my native
> >language) is the way I handle the two types of tuples add_package
> >needs to process. During registration the tuple contains a arch
> >and a cookie, but in later updates on these two fields are missing.
> >I used a boolean and a try/except to switch between the two. There
> >might be a smarter way of handeling this.
>
> Hmmm...I forget what the cookie is. I probably have that written down
> someplace. You might want to use len(tuple) and do this based off how
> many elements you find in the tuple. I'll look at it in more detail
> later.

Cookie is what the docs call 'buildhost number' or something. I believe
the RPM_tag calls it a cookie.

> >Some TODO's, which i'm willing to work on:
> > - crypt/md5 the password before putting it in database or using it.
>
> SHA1 I think is best.

It's just used for a making unreadable of the password. (hash)

> > - store hardware info in HARDWARE table. Need better format than
> > current table scheme.
>
> Not high priority....the database info that up2date shoves at Current
> is fun to handle. A bunch of random dicts. The only commonality is
> that they all have a 'class' field.

and a desc. But yes. nice to have. Esp. if we are going to have a
webinterface giving out of kind of neat information of the system.

> Need to figure out how (or if possible) we can do this with Yum.

I think you have a big ouch coming up.

> > - add some fields to PROFILE so we can check if the system logs
> > in, and if so, how often.
>
> Yes...needed soon. Also look at the webpage stuff that's now in
> trunk. I'm using the TurboGears toolkit.

Never heard of it. I will google some more at home.

> > - setup a QUEUE, so we can tickle a system to resend profile etc.
> > Queue itself is not difficult, all you need to do is sent a
> > properly formatted (python serialized) method to the client.
> > [i could invision a cadmin queue command for this :) ]
> >
>
> Again, need support for Yum. Red Hat may be workiing on a solution
> here as well.

We need to fix that when Yum has a standard for it. I presume you are not
in the working comittee of yum-standardisation? :)

> >Some nagging questions:
> > Why is PROFILE not called SYSTEM?
>
> Because I am horrible at keeping names consistant. :-) I imagine all
> profile/system things should refer to a "system".

Can we change this at some point? (Better soon than later). I believe svn
has good rename capabilities, so history wont get lost.

> > Why is INSTALLED not called SYSTEM_INSTALLED?
> > Why is HARDWARE not called SYSTEM_HARDWARE?

    Met vriendelijke groet,
        Pauline Middelink
--
GPG Key fingerprint = 2D5B 87A7 DDA6 0378 5DEA BD3B 9A50 B416 E2D0 C3C2
For more details look at my website http://www.polyware.​nl/~middelink
Attachments

Re: Patch against 1.7.4

Reply

Author Pauline Middelink <middelink at polyware dot nl>
Full name Pauline Middelink <middelink at polyware dot nl>
Date 2006-08-11 09:00:26 PDT
Message On Fri, 11 Aug 2006 around 09:31:08 -0400, Jack Neely wrote:
> Pauline,
>
> Umm...I think you forgot to include the users.py file. :-)

I'm afraid I forgot more. The exception handeling is syntaxicly
wrong :(

If you forgive me, here is a snipped code that does work.

(api/registration.py: line 200
----------
    for (pkg) in package_list:
        if first:
            try:
                (name,version,releas​e,epoch,arch,cookie)​ = pkg
            except Exception, e:
                first = 0
                (name,version,release,epoch) = pkg
        else:
            (name,version,release,epoch) = pkg

        p.addPackage(subscri​bedChans,name,versio​n,release,epoch)
    return 0
----------

I also found the problem people had getting a list of compatible
channels. In function setupBaseChannel, you need to compare
    PROFILE.os_release = CHANNEL.osrelease, not
    PROFILE.release_name = CHANNEL.osrelease

To make matters 'worse', I added a small function, getArch to
pysqlite, so I could compare not PROFILE.architecture, but
getArch(PROFILE.architecture), thereby having the basechannel
compatible with its compatible architectures.

I will send a complete patch when i'm home.

    Met vriendelijke groet,
        Pauline Middelink
--
GPG Key fingerprint = 2D5B 87A7 DDA6 0378 5DEA BD3B 9A50 B416 E2D0 C3C2
For more details look at my website http://www.polyware.​nl/~middelink
Attachments

Re: Patch against 1.7.4

Reply

Author theslack
Full name Jack Neely
Date 2006-08-11 06:31:08 PDT
Message Pauline,

Umm...I think you forgot to include the users.py file. :-)

Jack

On 8/11/06, Pauline Middelink <middelink at polyware dot nl> wrote:
> Hi Jack,
>
> Attached a patch for 1.7.4. I tried to add some user handeling to
> current.
>
> In essence it creates a USER table, with for now only a user_id,
> username and password. During registration up2date will check
> if the user/passwd already exists, and if not make a new user for
> it. This works. Then the patch adds a user_id to the profiles,
> making the systems 'owned' by the user. This seems to work in
> my little testcases ok.
>
> Second thing the patch does is implement functionality for the
> add/del/update package list which is maintained my up2date/rhn_check.
> It uses the INSTALLED database, which has been changed to include
> name,version,release and epoch fields. In my opinion there was
> a thinko in the previous design, it assumed all the packages the
> client has installed could be matched to a package_id. Does not
> work in the real-world :). So the package_id became optional.
> The patch tries to maintain a mapping from the INSTALLED.nvre
> tuple to the current PACKAGE list (during deleteRPMS and addPackage).
> Thus, for the packages which ARE known to current, the
> INSTALLED.package_id is/should be correct.
>
> !!!
> Something you might want to look at (python is not my native
> language) is the way I handle the two types of tuples add_package
> needs to process. During registration the tuple contains a arch
> and a cookie, but in later updates on these two fields are missing.
> I used a boolean and a try/except to switch between the two. There
> might be a smarter way of handeling this.
>
> Some TODO's, which i'm willing to work on:
> - crypt/md5 the password before putting it in database or using it.
> - store hardware info in HARDWARE table. Need better format than
> current table scheme.
> - add some fields to PROFILE so we can check if the system logs
> in, and if so, how often.
> - setup a QUEUE, so we can tickle a system to resend profile etc.
> Queue itself is not difficult, all you need to do is sent a
> properly formatted (python serialized) method to the client.
> [i could invision a cadmin queue command for this :) ]
>
> Some nagging questions:
> Why is PROFILE not called SYSTEM?
> Why is INSTALLED not called SYSTEM_INSTALLED?
> Why is HARDWARE not called SYSTEM_HARDWARE?
>
> Met vriendelijke groet,
> Pauline Middelink
> --
> GPG Key fingerprint = 2D5B 87A7 DDA6 0378 5DEA BD3B 9A50 B416 E2D0 C3C2
> For more details look at my website http://www.polyware.​nl/~middelink
>
>
>

Re: Patch against 1.7.4

Reply

Author theslack
Full name Jack Neely
Date 2006-08-11 06:20:45 PDT
Message Pauline,

Abosultely awesome. Thanks a bunch. I've actually been struggling
with how to handle user authentication for a while. You'd think this
is common and there would be libraries or best practices or something
similarly googlable.

Let's see...if you handle an exception in most cases you also want to
call logException() in the except clause so that the exception get in
the log file.

I'm not sure I want to implenent random creation of new users via
up2date's registration thing. But that's very minor and the code is
definitely needed. One of the bigger things I'm aiming for is to
bring what RHN should be with the up2date client to Yum. (Especially
since up2date is going away.) In the long run that probably means a
replacement for rhn_check. But we might be able to get some code from
Red Hat's RHN Yum plugin when that is available.

Hmmm...I probably haven't explained well to my user/devel community my
"vision" (*chuckle*) for supporting Yum.

Few comments inline.

This is fantastic. Thanks a bunch!

Jack

On 8/11/06, Pauline Middelink <middelink at polyware dot nl> wrote:
> Hi Jack,
>
> Attached a patch for 1.7.4. I tried to add some user handeling to
> current.
>
> In essence it creates a USER table, with for now only a user_id,
> username and password. During registration up2date will check
> if the user/passwd already exists, and if not make a new user for
> it. This works. Then the patch adds a user_id to the profiles,
> making the systems 'owned' by the user. This seems to work in
> my little testcases ok.
>

*Happy Dance*

> Second thing the patch does is implement functionality for the
> add/del/update package list which is maintained my up2date/rhn_check.
> It uses the INSTALLED database, which has been changed to include
> name,version,release and epoch fields. In my opinion there was
> a thinko in the previous design, it assumed all the packages the
> client has installed could be matched to a package_id. Does not
> work in the real-world :). So the package_id became optional.
> The patch tries to maintain a mapping from the INSTALLED.nvre
> tuple to the current PACKAGE list (during deleteRPMS and addPackage).
> Thus, for the packages which ARE known to current, the
> INSTALLED.package_id is/should be correct.

You're correct. The thinko was one reason (besides the whole working
for a living) that the INSTALLED table stuff was not implemented.

Your database foo may be stronger than mine. Does it make since to
have an unknown packages table and make INSTALLED a mapping between
profile_ids and package_id or unknownpkg_id?

>
> !!!
> Something you might want to look at (python is not my native
> language) is the way I handle the two types of tuples add_package
> needs to process. During registration the tuple contains a arch
> and a cookie, but in later updates on these two fields are missing.
> I used a boolean and a try/except to switch between the two. There
> might be a smarter way of handeling this.

Hmmm...I forget what the cookie is. I probably have that written down
someplace. You might want to use len(tuple) and do this based off how
many elements you find in the tuple. I'll look at it in more detail
later.

>
> Some TODO's, which i'm willing to work on:
> - crypt/md5 the password before putting it in database or using it.

SHA1 I think is best.

> - store hardware info in HARDWARE table. Need better format than
> current table scheme.

Not high priority....the database info that up2date shoves at Current
is fun to handle. A bunch of random dicts. The only commonality is
that they all have a 'class' field.

Need to figure out how (or if possible) we can do this with Yum.

> - add some fields to PROFILE so we can check if the system logs
> in, and if so, how often.

Yes...needed soon. Also look at the webpage stuff that's now in
trunk. I'm using the TurboGears toolkit.

> - setup a QUEUE, so we can tickle a system to resend profile etc.
> Queue itself is not difficult, all you need to do is sent a
> properly formatted (python serialized) method to the client.
> [i could invision a cadmin queue command for this :) ]
>

Again, need support for Yum. Red Hat may be workiing on a solution
here as well.

> Some nagging questions:
> Why is PROFILE not called SYSTEM?

Because I am horrible at keeping names consistant. :-) I imagine all
profile/system things should refer to a "system".

> Why is INSTALLED not called SYSTEM_INSTALLED?
> Why is HARDWARE not called SYSTEM_HARDWARE?
>
> Met vriendelijke groet,
> Pauline Middelink
> --
> GPG Key fingerprint = 2D5B 87A7 DDA6 0378 5DEA BD3B 9A50 B416 E2D0 C3C2
> For more details look at my website http://www.polyware.​nl/~middelink

Thanks!!

Patch against 1.7.4

Reply

Author Pauline Middelink <middelink at polyware dot nl>
Full name Pauline Middelink <middelink at polyware dot nl>
Date 2006-08-11 03:42:51 PDT
Message Hi Jack,

Attached a patch for 1.7.4. I tried to add some user handeling to
current.

In essence it creates a USER table, with for now only a user_id,
username and password. During registration up2date will check
if the user/passwd already exists, and if not make a new user for
it. This works. Then the patch adds a user_id to the profiles,
making the systems 'owned' by the user. This seems to work in
my little testcases ok.

Second thing the patch does is implement functionality for the
add/del/update package list which is maintained my up2date/rhn_check.
It uses the INSTALLED database, which has been changed to include
name,version,release and epoch fields. In my opinion there was
a thinko in the previous design, it assumed all the packages the
client has installed could be matched to a package_id. Does not
work in the real-world :). So the package_id became optional.
The patch tries to maintain a mapping from the INSTALLED.nvre
tuple to the current PACKAGE list (during deleteRPMS and addPackage).
Thus, for the packages which ARE known to current, the
INSTALLED.package_id is/should be correct.

!!!
Something you might want to look at (python is not my native
language) is the way I handle the two types of tuples add_package
needs to process. During registration the tuple contains a arch
and a cookie, but in later updates on these two fields are missing.
I used a boolean and a try/except to switch between the two. There
might be a smarter way of handeling this.

Some TODO's, which i'm willing to work on:
 - crypt/md5 the password before putting it in database or using it.
 - store hardware info in HARDWARE table. Need better format than
   current table scheme.
 - add some fields to PROFILE so we can check if the system logs
   in, and if so, how often.
 - setup a QUEUE, so we can tickle a system to resend profile etc.
   Queue itself is not difficult, all you need to do is sent a
   properly formatted (python serialized) method to the client.
   [i could invision a cadmin queue command for this :) ]

Some nagging questions:
  Why is PROFILE not called SYSTEM?
  Why is INSTALLED not called SYSTEM_INSTALLED?
  Why is HARDWARE not called SYSTEM_HARDWARE?

    Met vriendelijke groet,
        Pauline Middelink
--
GPG Key fingerprint = 2D5B 87A7 DDA6 0378 5DEA BD3B 9A50 B416 E2D0 C3C2
For more details look at my website http://www.polyware.​nl/~middelink
Attachments
Messages per page: