From geeklog at langfamily.ca Wed Mar 3 00:04:55 2004 From: geeklog at langfamily.ca (Blaine Lang) Date: Wed, 3 Mar 2004 00:04:55 -0500 Subject: [geeklog-devel] resolution to an ODD CSS issue Message-ID: <001901c400dd$116cec60$650a10ac@XPBL2> While developing a CSS menu, I was having a real problem trying to get rid of a spacing issue that IE 6 was having. My problem went away after adding the URI to the doctype specification in header.thtml for the XSilver theme. My problem actually goes away by just adding "http://www.w3.org". Apparently with No URI present, IE 6 falls into "quirky" mode and was not interpreting the CSS correctly. I am still learning CSS and SGML so maybe others want to comment on this and if we should be declaring the URI in the theme header.thtm files. I was using XSilver but it appears all the themes are the same. This is an MS Document that provides more information. http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnie60/html/cssenhancements.asp Blaine -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony at tonybibbs.com Wed Mar 3 10:12:41 2004 From: tony at tonybibbs.com (Tony Bibbs) Date: Wed, 03 Mar 2004 09:12:41 -0600 Subject: [geeklog-devel] Re: [geeklog-devtalk] Variable best practices In-Reply-To: <002701c40127$b01bf0f0$650a10ac@XPBL2> References: <002601c3fed7$5bcfe1c0$650a10ac@XPBL2> <001501c3fef0$8dedbd80$650a10ac@XPBL2> <002701c40127$b01bf0f0$650a10ac@XPBL2> Message-ID: <4045F5E9.9050308@tonybibbs.com> Knowing that there have been security issues with anything prior to 4.2 (and, indeed, after), I don't think it is a stretch to have Geeklog require 4.1 or better. In fact, we should probably have a rule that Geeklog will support so many revisions behind the most current. I mean, seriously, should we be concerned with people who are still using pre-4.1 (hell, the 5.0 launch isn't all that far away)? I think not....simply because I can't fatham that many people out there with that old of a version. Sure, that means you have some inconsistent code (i.e. some that can handle no register_globals and some that can) but to me that at least introduces the ability to start making the switch and, at the very least, encourages good coding habits. As you can tell, I have an opinion. I say ditch pre-4.1 support and let's start using $_REQUEST, $_GET, $_POST and be happy. I, of course, could be swayed if there is enough uproar to convince me pre-4.1 support is needed. Thoughts? --Tony Blaine Lang wrote: > Maybe, I can only receive emails on this list ;) > > Another suggestion where this function would be helpfull is in say ..... > admin/topic.php -- I believe $tid and $topic are variables that are > passed as GET or POST vars. > > You need to check for both, post is higher in presidence and if not set > it to ''; > > Just a thought -- I'll stop sharing now. > > ----- Original Message ----- > *From:* Blaine Lang > *To:* geeklog-devtalk at lists.geeklog.net > > *Sent:* Sunday, February 29, 2004 1:19 PM > *Subject:* Re: [geeklog-devtalk] Variable best practices > > I've been wanting to address this issue for a while and have come up > with the following function. > > It will filter either GET, POST or both (as in my need noted below - > but look for POST first). > It will appy the data filter - default mode for now. > It will also optionally create GLOBALS out of the variables. This > could be used to remove a dependany on Register_globals. > > Example use: > > $myvars = array('op','profileid','memberid'); > getdata($myvars,true); > > or > > $formvars = array('profile_name','member_num', 'member_addr', > 'member_city', 'member_phone'); > $formdata = array(); > $formdata = getdata($formvars,'POST'); > > << code begin >> > > function getdata($vars,$setglobal=false,$type='') { > $return_data = array(); > > #setup common reference to SuperGlobals depending which array is > needed > if ($type == "GET" OR $type == "POST") { > if ($type=="GET") { $SG_Array=& $_GET; } > if ($type=="POST") { $SG_Array=& $_POST; } > > # loop through SuperGlobal data array and grab out data for > allowed fields if found > foreach($vars as $key) { > if (array_key_exists($key,$SG_Array)) { > $return_data[$key]=$SG_Array[$key]; } > } > > } else { > foreach ($vars as $key) { > if (array_key_exists($key, $_POST)) { > $return_data[$key] = $_POST[$key]; > } elseif (array_key_exists($key, $_GET)) { > $return_data[$key] = $_GET[$key]; > } > } > } > > # loop through $vars array and apply the filter > foreach($vars as $value) { > $return_data[$value] = COM_applyFilter($return_data[$value]); > } > > // Optionally set $GLOBALS or return the array > if ($setglobal) { > # loop through final data and define all the variables using > the $GLOBALS array > foreach ($return_data as $key=>$value) { > $GLOBALS[$key]=$value; > } > } else { > return $return_data; > } > > } > > > > ----- Original Message ----- > *From:* Blaine Lang > *To:* geeklog-devtalk at lists.geeklog.net > > *Sent:* Sunday, February 29, 2004 10:19 AM > *Subject:* [geeklog-devtalk] Variable best practices > > As a practce, I try not to have any register_globals > dependancies. I will use the full $HTTP_POST_VARS > and $HTTP_GET_VARS variable names. I'd lole to use the other > supper globals ($_REQUEST, $_POST and $_GET) and are still > dependant on php version 4.1 or greater. Most hosting services > should be running by now but it can not be assumed I guess. > > I'm filtering all expected POST and GET vars now in my new > plugins and there are still times it makes sense to use the same > name in the POST and GET. It may be passed in the URL or form so > you have to check both but I check the post first as I would use > that first if I can post a hidden variable. > > Examples are: > using $op to indicate operation. I may be triggered from a link, > selectbox, image or button in the UI. I may also have to pass it > to another script. > using $page and $sort as part of the page google like navigation > (links) but you also need to retain this as hidden fields in > forms if you want to retain your page position. > > I don't see the reason to use different names with the get and > post as this allows me more control. I only expect it one way or > the other and POST should override get. > > Hense why I end of having all this extra code at the top of scripts: > > if (isset($HTTP_POST_VARS['op']) ) { > > $op = clubApplyFilter($HTTP_POST_VARS['op']); > > } elseif (isset($HTTP_GET_VARS['op']) ) { > > $op = clubApplyFilter($HTTP_GET_VARS['op']); > > } else { > > $op = ''; > > } > > Is this how others would approach this? > > I was thinking if I am having to repeat this practice and common > code, others would as well. Extending my class object to handle > this would be an advantage. > $myfilter = new COM_filter; > $myfilter->_censor = true; > $myfilter ->_jsfilter = true; > $op = $myfilter->_setvar('op', 'text'); > > Cheers, > Blaine > > From geeklog at langfamily.ca Wed Mar 3 10:59:43 2004 From: geeklog at langfamily.ca (Blaine Lang) Date: Wed, 3 Mar 2004 10:59:43 -0500 Subject: [geeklog-devel] Re: [geeklog-devtalk] Variable best practices References: <002601c3fed7$5bcfe1c0$650a10ac@XPBL2> <001501c3fef0$8dedbd80$650a10ac@XPBL2> <002701c40127$b01bf0f0$650a10ac@XPBL2> <4045F5E9.9050308@tonybibbs.com> Message-ID: <005a01c40138$8b2b3ec0$650a10ac@XPBL2> Tony, I don't disagree and using $_REQUEST would remove the need to check both GET and POST. I wonder though, if it is not still more secure to check for $_POST before $_GET. I know a hacker can just as easily fake a POST request so maybe it does not matter. As long as we are filtering out any potentially hostile data. I suspect our issues are mostly with users that host their sites. That still leaves the question or feedback and using the concept I used in this function for setting up which variables you want to filter and calling the filter once. It also has a mode to address the need for Register_Globals on. Blaine ----- Original Message ----- From: "Tony Bibbs" To: Sent: Wednesday, March 03, 2004 10:12 AM Subject: [geeklog-devel] Re: [geeklog-devtalk] Variable best practices > Knowing that there have been security issues with anything prior to 4.2 > (and, indeed, after), I don't think it is a stretch to have Geeklog > require 4.1 or better. In fact, we should probably have a rule that > Geeklog will support so many revisions behind the most current. I mean, > seriously, should we be concerned with people who are still using > pre-4.1 (hell, the 5.0 launch isn't all that far away)? I think > not....simply because I can't fatham that many people out there with > that old of a version. Sure, that means you have some inconsistent code > (i.e. some that can handle no register_globals and some that can) but to > me that at least introduces the ability to start making the switch and, > at the very least, encourages good coding habits. > > As you can tell, I have an opinion. I say ditch pre-4.1 support and > let's start using $_REQUEST, $_GET, $_POST and be happy. I, of course, > could be swayed if there is enough uproar to convince me pre-4.1 support > is needed. > > Thoughts? > > --Tony > > Blaine Lang wrote: > > Maybe, I can only receive emails on this list ;) > > > > Another suggestion where this function would be helpfull is in say ..... > > admin/topic.php -- I believe $tid and $topic are variables that are > > passed as GET or POST vars. > > > > You need to check for both, post is higher in presidence and if not set > > it to ''; > > > > Just a thought -- I'll stop sharing now. > > > > ----- Original Message ----- > > *From:* Blaine Lang > > *To:* geeklog-devtalk at lists.geeklog.net > > > > *Sent:* Sunday, February 29, 2004 1:19 PM > > *Subject:* Re: [geeklog-devtalk] Variable best practices > > > > I've been wanting to address this issue for a while and have come up > > with the following function. > > > > It will filter either GET, POST or both (as in my need noted below - > > but look for POST first). > > It will appy the data filter - default mode for now. > > It will also optionally create GLOBALS out of the variables. This > > could be used to remove a dependany on Register_globals. > > > > Example use: > > > > $myvars = array('op','profileid','memberid'); > > getdata($myvars,true); > > > > or > > > > $formvars = array('profile_name','member_num', 'member_addr', > > 'member_city', 'member_phone'); > > $formdata = array(); > > $formdata = getdata($formvars,'POST'); > > > > << code begin >> > > > > function getdata($vars,$setglobal=false,$type='') { > > $return_data = array(); > > > > #setup common reference to SuperGlobals depending which array is > > needed > > if ($type == "GET" OR $type == "POST") { > > if ($type=="GET") { $SG_Array=& $_GET; } > > if ($type=="POST") { $SG_Array=& $_POST; } > > > > # loop through SuperGlobal data array and grab out data for > > allowed fields if found > > foreach($vars as $key) { > > if (array_key_exists($key,$SG_Array)) { > > $return_data[$key]=$SG_Array[$key]; } > > } > > > > } else { > > foreach ($vars as $key) { > > if (array_key_exists($key, $_POST)) { > > $return_data[$key] = $_POST[$key]; > > } elseif (array_key_exists($key, $_GET)) { > > $return_data[$key] = $_GET[$key]; > > } > > } > > } > > > > # loop through $vars array and apply the filter > > foreach($vars as $value) { > > $return_data[$value] = COM_applyFilter($return_data[$value]); > > } > > > > // Optionally set $GLOBALS or return the array > > if ($setglobal) { > > # loop through final data and define all the variables using > > the $GLOBALS array > > foreach ($return_data as $key=>$value) { > > $GLOBALS[$key]=$value; > > } > > } else { > > return $return_data; > > } > > > > } > > > > > > > > ----- Original Message ----- > > *From:* Blaine Lang > > *To:* geeklog-devtalk at lists.geeklog.net > > > > *Sent:* Sunday, February 29, 2004 10:19 AM > > *Subject:* [geeklog-devtalk] Variable best practices > > > > As a practce, I try not to have any register_globals > > dependancies. I will use the full $HTTP_POST_VARS > > and $HTTP_GET_VARS variable names. I'd lole to use the other > > supper globals ($_REQUEST, $_POST and $_GET) and are still > > dependant on php version 4.1 or greater. Most hosting services > > should be running by now but it can not be assumed I guess. > > > > I'm filtering all expected POST and GET vars now in my new > > plugins and there are still times it makes sense to use the same > > name in the POST and GET. It may be passed in the URL or form so > > you have to check both but I check the post first as I would use > > that first if I can post a hidden variable. > > > > Examples are: > > using $op to indicate operation. I may be triggered from a link, > > selectbox, image or button in the UI. I may also have to pass it > > to another script. > > using $page and $sort as part of the page google like navigation > > (links) but you also need to retain this as hidden fields in > > forms if you want to retain your page position. > > > > I don't see the reason to use different names with the get and > > post as this allows me more control. I only expect it one way or > > the other and POST should override get. > > > > Hense why I end of having all this extra code at the top of scripts: > > > if (isset($HTTP_POST_VARS['op']) ) { > > > $op = clubApplyFilter($HTTP_POST_VARS['op']); > > > } elseif (isset($HTTP_GET_VARS['op']) ) { > > > $op = clubApplyFilter($HTTP_GET_VARS['op']); > > > } else { > > > $op = ''; > > > } > > > > Is this how others would approach this? > > > > I was thinking if I am having to repeat this practice and common > > code, others would as well. Extending my class object to handle > > this would be an advantage. > > $myfilter = new COM_filter; > > $myfilter->_censor = true; > > $myfilter ->_jsfilter = true; > > $op = $myfilter->_setvar('op', 'text'); > > > > Cheers, > > Blaine > > > > > _______________________________________________ > geeklog-devel mailing list > geeklog-devel at lists.geeklog.net > http://lists.geeklog.net/listinfo/geeklog-devel From tony at tonybibbs.com Wed Mar 3 14:38:58 2004 From: tony at tonybibbs.com (Tony Bibbs) Date: Wed, 03 Mar 2004 13:38:58 -0600 Subject: [geeklog-devel] Re: [geeklog-devtalk] Variable best practices In-Reply-To: <005a01c40138$8b2b3ec0$650a10ac@XPBL2> References: <002601c3fed7$5bcfe1c0$650a10ac@XPBL2> <001501c3fef0$8dedbd80$650a10ac@XPBL2> <002701c40127$b01bf0f0$650a10ac@XPBL2> <4045F5E9.9050308@tonybibbs.com> <005a01c40138$8b2b3ec0$650a10ac@XPBL2> Message-ID: <40463452.9030203@tonybibbs.com> > Tony, > > I don't disagree and using $_REQUEST would remove the need to check both GET > and POST. > I wonder though, if it is not still more secure to check for $_POST before > $_GET. I know a hacker can just as easily fake a POST request so maybe it > does not matter. As long as we are filtering out any potentially hostile > data. Right, see injections can happen, in theory any time you go to the database (via a select, update, etc it doesn't matter). So those fields should be filtered. How you do that doesn't matter and my hunch is just going after the data in $_REQUEST is good enough as it handles everything. Now, in Geeklog 2 we are using PEAR and thus I am using prepared SQL statements: $prepStmt = $_DB->prepare('SELECT * FROM USERS WHERE uid = ?'); $result = $_DB->execute($prepStmt,$_REQUEST['uid']); To handle injection attempts I plan on creating a method called DB_execute() that mirrors the PEAR execute method above except I will pass all arguments through the filter which should provide a convenient way of filtering stuff without having to have to explicitly do it every time. Anyway, just wanted to clue you in on how I have been thinking through all this. For 1.3.x your function is good and the only drawback is you have to call it explicitly but I don't see a way around it. > > I suspect our issues are mostly with users that host their sites. > Yeah, and again, most sane hosts should be on 4.1 or better already. > > It also has a mode to address the need for Register_Globals on. > There is a lot that would need to be done to 1.3.x to get rid of this but if you move forward assuming 4.1. or better at least we can mark each page as a to-do in terms of getting rid of that dependency. Then again, maybe this should be somthing similar to when I introduced HTML templates where we plan on freezing the code and working on nothing but register_globals issues and releasing that under 1.4.x. That is something for Dirk to ponder but I'd be willing to help knock out a few pages. With three of us we might actually get a beta version of this out within a couple of weeks. --Tony From dirk at haun-online.de Fri Mar 5 18:10:27 2004 From: dirk at haun-online.de (Dirk Haun) Date: Sat, 6 Mar 2004 00:10:27 +0100 Subject: [geeklog-devel] Release Candidate 3? Message-ID: <20040305231027.23464@smtp.haun-online.de> I'm tinking about releasing a third release candidate this weekend, followed by the final version one week later (say, on the 15th). Is anyone aware of any remaining issues that need to be fixed? bye, Dirk -- http://www.haun-online.de/ http://www.macosx-faq.de/ From tony at tonybibbs.com Mon Mar 8 09:05:02 2004 From: tony at tonybibbs.com (Tony Bibbs) Date: Mon, 08 Mar 2004 08:05:02 -0600 Subject: [geeklog-devel] Regex functions in Geeklog Message-ID: <404C7D8E.3080703@tonybibbs.com> An anonymous coward wrote: I was looking through the Geeklog PHP code and saw it uses functions like eregi and ereg_replace in places, when it should be using preg_match or str_replace, which are much faster. I've been coding in PHP since '99 and I've benchmarked the preg and string functions against the ereg functions and the ereg functions are always slower and should never be used. You should use strstr or stristr instead of ereg or preg_match if you don't need a regular expression. And you should use str_replace instead of preg_replace or ereg_replace if you don't need a regular expression. Please consider making these changes in future versions of the code. I've forward this to the list as an FYI. This guy isn't saying something we haven't heard before. BTW, for anybody watching this list, anonymously sent emails annoy me (and other developers I'm sure). Use geeklog-devtalk or geeklog-users for stuff like this. --Tony From tony at tonybibbs.com Mon Mar 8 12:33:18 2004 From: tony at tonybibbs.com (Tony Bibbs) Date: Mon, 08 Mar 2004 11:33:18 -0600 Subject: [geeklog-devel] Regex functions in Geeklog In-Reply-To: References: <404C7D8E.3080703@tonybibbs.com> Message-ID: <404CAE5E.9040904@tonybibbs.com> Yep, an RC3 is out now. I doubt you get the fixes in for the final 1.3.9 release but we can try (Dirk, you have an ETA yet?). Just send a note to geeklog-devel once you think you got it. --Tony Justin Carlson wrote: > Is it ereg_replace or eregi_replace? > I will assume that since this guy has been coding since `99 that it is just > ereg_replace(). > > I could possibly make the adjustments. > You guys have a RC getting ready to be put out right? > > ----- Original Message ----- > From: "Tony Bibbs" > To: "Geeklog" > Sent: Monday, March 08, 2004 8:05 AM > Subject: [geeklog-devel] Regex functions in Geeklog > > > >>An anonymous coward wrote: >> >> >>I was looking through the Geeklog PHP code and saw it uses functions >>like eregi and ereg_replace in places, when it should be using >>preg_match or str_replace, which are much faster. I've been coding in >>PHP since '99 and I've benchmarked the preg and string functions against >>the ereg functions and the ereg functions are always slower and should >>never be used. You should use strstr or stristr instead of ereg or >>preg_match if you don't need a regular expression. And you should use >>str_replace instead of preg_replace or ereg_replace if you don't need a >>regular expression. >> >>Please consider making these changes in future versions of the code. >> >> >>I've forward this to the list as an FYI. This guy isn't saying >>something we haven't heard before. BTW, for anybody watching this list, >>anonymously sent emails annoy me (and other developers I'm sure). Use >>geeklog-devtalk or geeklog-users for stuff like this. >> >>--Tony >>_______________________________________________ >>geeklog-devel mailing list >>geeklog-devel at lists.geeklog.net >>http://lists.geeklog.net/listinfo/geeklog-devel >> From geeklog at langfamily.ca Mon Mar 8 12:40:52 2004 From: geeklog at langfamily.ca (Blaine Lang) Date: Mon, 8 Mar 2004 12:40:52 -0500 Subject: [geeklog-devel] Regex functions in Geeklog References: <404C7D8E.3080703@tonybibbs.com> <404CAE5E.9040904@tonybibbs.com> Message-ID: <00d101c40534$80aa9e00$650a10ac@XPBL2> I would personally hold off any new changes and I think Dirk will also feel the same. Only bug fixes to 1.3.9 changes should be done at this time. We can discuss making the regex function changes for 1.3.10 Blaine ----- Original Message ----- From: "Tony Bibbs" To: "Geeklog" ; Sent: Monday, March 08, 2004 12:33 PM Subject: Re: [geeklog-devel] Regex functions in Geeklog > Yep, an RC3 is out now. I doubt you get the fixes in for the final > 1.3.9 release but we can try (Dirk, you have an ETA yet?). Just send a > note to geeklog-devel once you think you got it. > > --Tony > > Justin Carlson wrote: > > Is it ereg_replace or eregi_replace? > > I will assume that since this guy has been coding since `99 that it is just > > ereg_replace(). > > > > I could possibly make the adjustments. > > You guys have a RC getting ready to be put out right? > > > > ----- Original Message ----- > > From: "Tony Bibbs" > > To: "Geeklog" > > Sent: Monday, March 08, 2004 8:05 AM > > Subject: [geeklog-devel] Regex functions in Geeklog > > > > > > > >>An anonymous coward wrote: > >> > >> > >>I was looking through the Geeklog PHP code and saw it uses functions > >>like eregi and ereg_replace in places, when it should be using > >>preg_match or str_replace, which are much faster. I've been coding in > >>PHP since '99 and I've benchmarked the preg and string functions against > >>the ereg functions and the ereg functions are always slower and should > >>never be used. You should use strstr or stristr instead of ereg or > >>preg_match if you don't need a regular expression. And you should use > >>str_replace instead of preg_replace or ereg_replace if you don't need a > >>regular expression. > >> > >>Please consider making these changes in future versions of the code. > >> > >> > >>I've forward this to the list as an FYI. This guy isn't saying > >>something we haven't heard before. BTW, for anybody watching this list, > >>anonymously sent emails annoy me (and other developers I'm sure). Use > >>geeklog-devtalk or geeklog-users for stuff like this. > >> > >>--Tony > >>_______________________________________________ > >>geeklog-devel mailing list > >>geeklog-devel at lists.geeklog.net > >>http://lists.geeklog.net/listinfo/geeklog-devel > >> > _______________________________________________ > geeklog-devel mailing list > geeklog-devel at lists.geeklog.net > http://lists.geeklog.net/listinfo/geeklog-devel From dirk at haun-online.de Mon Mar 8 14:27:29 2004 From: dirk at haun-online.de (Dirk Haun) Date: Mon, 8 Mar 2004 20:27:29 +0100 Subject: [geeklog-devel] Regex functions in Geeklog In-Reply-To: <00d101c40534$80aa9e00$650a10ac@XPBL2> References: <00d101c40534$80aa9e00$650a10ac@XPBL2> Message-ID: <20040308192729.9200@smtp.haun-online.de> >I would personally hold off any new changes and I think Dirk will also feel >the same. Yep. Also, I don't think replacing them would make a noticable difference. I'm open for suggestions, though (for future versions). bye, Dirk -- http://www.haun-online.de/ http://www.haun.info/ From tony at tonybibbs.com Mon Mar 8 16:46:03 2004 From: tony at tonybibbs.com (Tony Bibbs) Date: Mon, 08 Mar 2004 15:46:03 -0600 Subject: [geeklog-devel] Regex functions in Geeklog In-Reply-To: <20040308192729.9200@smtp.haun-online.de> References: <00d101c40534$80aa9e00$650a10ac@XPBL2> <20040308192729.9200@smtp.haun-online.de> Message-ID: <404CE99B.7090209@tonybibbs.com> No big deal...I'll work with Justin and when he gets it working I'll work with you Dirk, to get it into CVS, along with my spellchecking code which is finally ready for CVS. --Tony Dirk Haun wrote: >>I would personally hold off any new changes and I think Dirk will also feel >>the same. > > > Yep. Also, I don't think replacing them would make a noticable difference. > > I'm open for suggestions, though (for future versions). > > bye, Dirk > > From geeklog at langfamily.ca Tue Mar 9 22:55:10 2004 From: geeklog at langfamily.ca (Blaine Lang) Date: Tue, 9 Mar 2004 22:55:10 -0500 Subject: [geeklog-devel] Release Candidate 3? References: <20040305231027.23464@smtp.haun-online.de> Message-ID: <009401c40653$7bc72f00$650a10ac@XPBL2> Dirk, Did you see this post regarding the backup function. http://www.geeklog.net/forum/viewtopic.php?forum=3&showtopic=31718&lastpost=true I figure this should wait for 1.3.10 but its a good idea as it may be the source of others having similar issues. Blaine ----- Original Message ----- From: "Dirk Haun" To: Sent: Friday, March 05, 2004 6:10 PM Subject: [geeklog-devel] Release Candidate 3? > I'm tinking about releasing a third release candidate this weekend, > followed by the final version one week later (say, on the 15th). > > Is anyone aware of any remaining issues that need to be fixed? > > bye, Dirk > > > -- > http://www.haun-online.de/ > http://www.macosx-faq.de/ > > _______________________________________________ > geeklog-devel mailing list > geeklog-devel at lists.geeklog.net > http://lists.geeklog.net/listinfo/geeklog-devel From tony at tonybibbs.com Fri Mar 12 09:40:30 2004 From: tony at tonybibbs.com (Tony Bibbs) Date: Fri, 12 Mar 2004 08:40:30 -0600 Subject: [geeklog-devel] [Fwd: Re: Geeklog optimalisations] Message-ID: <4051CBDE.4010407@tonybibbs.com> This is an FYI. We'll be discussing this on the development lists over the next few days (I hope). It's important we help Groklaw as best we can as they are one of our bigger sites and by them pushing the limits of Geeklog we can address their issues and make Geeklog a better product at the same time. --Tony --------------------- Niels, I think a bit of background is in order before you can understand how Geeklog got where it is. First, nearly all the code you are referring to is legacy code. It was there before I managed the project and it is still there under Dirk's management. In it's infancy, Geeklog was only servicing smaller sites so performance was never really an issue and, frankly, I was a bit young and dumb when I first got started with Geeklog so performance tuning PHP scripts wasn't even a consideration and my focus was on the feature set. Under Dirk's management, the feature set has continued to grow to the point that we have a large userbase and what you are encountering with Geeklog is only natural. Groklaw is clearly one of the biggest sites to run Geeklog. I have posted questions to our mailing lists asking about performance issues realted to bigger Geeklog sites getting no responses back so your email was a pleasant surprise. The long and the short of it is Geeklog has matured to a point where bigger sites are using it and we pushing the performance limits it has. Geeklog's database interaction has always been an issue for me and is a large part why I have chosen to get a new codebase up (i.e. Geeklog 2) while the 1.3.x continues. You are right, we need to address the performance issues and given the amount of work you have put into troubleshooting Groklaw I think you can play a critical part in that. What I would like to do is see us work closely with you to begin addressing these issues. A starting point would be to have a place where we can install a development version of Groklaw's database somewhere where we can run tests. Dirk and I don't have access to a database of that size and while we could fudge together some data using a real world example would sure be nice. Once we have a test bed, I'd be open to suggestions on how we might work on this to resolve your immediate issues *and* begin addressing performance tuning as a whole. #geeklog is where I dwell (though not always at the keyboard). If possible I'd like to see us discuss this on geeklog-devtalk. Niels, if you could join that list at http://lists.geeklog.net/listinfo we can carry this on there. In the meantime if you can catch Dirk or myself in IRC feel free to do so. FYI I'm out of town this weekend (FWIW I'm GMT -6) so I may not seem too responsive until I get back on Sunday. Thanks for contacting us, I'm sure we can address these issues. --Tony Niels Leenheer wrote: > Hi guys, > > First of all. What were you guys thinking? Sorry to be so rude, but I simply > had to get that off my chest. I feel better now. I'm okay. Really. > > As some of you may be aware of Groklaw is using Geeklog. It has turned in to > quite a busy website and stories with more than 700 comments are not out of > the ordinary. In addition to this being slashdotted has become normal. This > is where the problems started. The server can't handle much more. On busy > days the website turns into a crawling slow pile of .. > > As a regular reader and volunteer of Groklaw I offered to take a look at the > Geeklog source code and try to find some places that could benefit from > optimalisation. After some testing I've noticed that most of the problems > are due to load on the database server. > > The first thing I started working on is the code that generates all the > comments. It turns out that for every comment at least two queries are > executed. For a story with more than 700 comments this would mean more than > almost 1500 queries to generate the page. > > I've modified this code extensively and now we use one query to fetch all > the user details of all the people involved in posting. One query is used to > fetch all the comments that have no parent. One query to fetch all the > comments to do have parents. And if needed, one query to fetch the parent. > All this data is then turned into one big nested array, which is passed by > reference to the functions that actually print the data. Depending on how > many comments there are this could result in a speed improvement of about > 0% - 1000%. As you can imagine if you only have about 10 comments it would > not mean much, with 500 comments it would reduce the amount of queries > needed by about a 1000. It's a very big improvement. > > One other problem I've identified is table locking of the story table. The > statistics are stored in the same table as the actual content of the story. > So each time a story is displayed, it will use an UPDATE query and a SELECT > query on the same table. With a lot of requests the table is constantly > locked by the UPDATE queries and the SELECT queries are waiting. We've > disabled the statistics for now, but we are investigating the possibility of > moving the statistics to a separate table. > > Next is the database layer. The mysql_fetch_array() function has two > arguments. The second determines what the function returns. Either an > associative array, a numbered array or both. By default the function returns > both. This is what Geeklog does. In most of the code only the associative > array is used. Only in a couple of small instances the code requires an > numbered array. What we have done is to instruct the mysql_fetch_array() > function to return only an associative array by default. Only when the code > requires a numbered array we request both. This should lower the amount of > memory needed by Geeklog. > > The SEC_getUserGroups() function is also quite expensive. It is called > throughout the generation a page and it does not cache the information. We' > ve added a simple cache for the data that is fetched from the database which > eliminates another 30 or so queries. > > Next is the index page. The COM_featuredCheck() function is executed every > time the frontpage is requested. I've changed the loop that actually > displays the stories on the frontpage and included a check to see if there > is more than one featured story. If there is, the second story is not > displayed as such and the featuredCheck() function is called. This again > saves a couple of queries and the end result is the same. > > We are also using the mycal extension which I've almost completely > rewritten. Mycal uses a query for every day that is displayed and after my > modifications it only uses one query. A 27-34 reduction in queries. > > Now back to my first paragraph. I was pretty impressed with how easy it was > to get used to the way everything works in Geeklog. It was pretty easy to > understand and it looks like it was designed pretty well. But I was also > horrified when I saw the enormous amount of queries that are used, but I > guess Geeklog wasn't really designed with this kind of traffic and these > enormous amounts of comments in mind. > > Most of the changes we've made are not yet running on the production server. > Once we've properly tested everything and everything is stable, I'm willing > to look at how we can give these changes back to Geeklog. As simple patch > between the current version of Geeklog and Groklaw will be difficult, > because we are using 1.3.8-1sr4 and it also includes a lot of Groklaw > specific modifications. If you are interested in these modifications, please > let me know and we'll work something out. > > If you want to talk to me about this you can e-mail. In addition to this > I'll try to visit #geeklog as often as I can. > > Niels Leenheer > -- project manager phpAdsNew > > > From vmf at abtech.org Fri Mar 12 10:36:57 2004 From: vmf at abtech.org (Vincent Furia) Date: Fri, 12 Mar 2004 10:36:57 -0500 Subject: [geeklog-devel] [Fwd: Re: Geeklog optimalisations] In-Reply-To: <4051CBDE.4010407@tonybibbs.com> References: <4051CBDE.4010407@tonybibbs.com> Message-ID: <4051D919.10802@abtech.org> I'm in! Though if we're going to get serious about upgrading geeklog 1.3.x to reduce queries and seriously improve performance for high traffic sites it might be time to start thinking about branching 1.3.x off and using the main branch to start implementing these performance improvements (and perhaps calling it 1.4.x). I actually don't think we're too far away from getting Geeklog to decent speeds. As Niels pointed out (and as has been pointed out before I think) we only really have one table locking issue. The database queries are something that can be fixed with some intelligent caching and better shaped queries. If we're feeling really brave, swithing template systems to a pre-compiled template system like Smarty wouldn't be a bad idea. I think that will do ALOT to improve performance. -Vinny Tony Bibbs wrote: > This is an FYI. We'll be discussing this on the development lists over > the next few days (I hope). It's important we help Groklaw as best we > can as they are one of our bigger sites and by them pushing the limits > of Geeklog we can address their issues and make Geeklog a better product > at the same time. > > --Tony > > --------------------- > > Niels, > > I think a bit of background is in order before you can understand how > Geeklog got where it is. First, nearly all the code you are referring > to is legacy code. It was there before I managed the project and it is > still there under Dirk's management. In it's infancy, Geeklog was only > servicing smaller sites so performance was never really an issue and, > frankly, I was a bit young and dumb when I first got started with > Geeklog so performance tuning PHP scripts wasn't even a consideration > and my focus was on the feature set. > > Under Dirk's management, the feature set has continued to grow to the > point that we have a large userbase and what you are encountering with > Geeklog is only natural. Groklaw is clearly one of the biggest sites to > run Geeklog. I have posted questions to our mailing lists asking about > performance issues realted to bigger Geeklog sites getting no responses > back so your email was a pleasant surprise. > > The long and the short of it is Geeklog has matured to a point where > bigger sites are using it and we pushing the performance limits it has. > Geeklog's database interaction has always been an issue for me and is > a large part why I have chosen to get a new codebase up (i.e. Geeklog 2) > while the 1.3.x continues. You are right, we need to address the > performance issues and given the amount of work you have put into > troubleshooting Groklaw I think you can play a critical part in that. > > What I would like to do is see us work closely with you to begin > addressing these issues. A starting point would be to have a place > where we can install a development version of Groklaw's database > somewhere where we can run tests. Dirk and I don't have access to a > database of that size and while we could fudge together some data using > a real world example would sure be nice. Once we have a test bed, I'd > be open to suggestions on how we might work on this to resolve your > immediate issues *and* begin addressing performance tuning as a whole. > > #geeklog is where I dwell (though not always at the keyboard). If > possible I'd like to see us discuss this on geeklog-devtalk. Niels, if > you could join that list at http://lists.geeklog.net/listinfo we can > carry this on there. In the meantime if you can catch Dirk or myself in > IRC feel free to do so. FYI I'm out of town this weekend (FWIW I'm GMT > -6) so I may not seem too responsive until I get back on Sunday. > > Thanks for contacting us, I'm sure we can address these issues. > > --Tony > > > > Niels Leenheer wrote: > >> Hi guys, >> >> First of all. What were you guys thinking? Sorry to be so rude, but I >> simply >> had to get that off my chest. I feel better now. I'm okay. Really. >> >> As some of you may be aware of Groklaw is using Geeklog. It has turned >> in to >> quite a busy website and stories with more than 700 comments are not >> out of >> the ordinary. In addition to this being slashdotted has become normal. >> This >> is where the problems started. The server can't handle much more. On busy >> days the website turns into a crawling slow pile of .. >> >> As a regular reader and volunteer of Groklaw I offered to take a look >> at the >> Geeklog source code and try to find some places that could benefit from >> optimalisation. After some testing I've noticed that most of the problems >> are due to load on the database server. >> >> The first thing I started working on is the code that generates all the >> comments. It turns out that for every comment at least two queries are >> executed. For a story with more than 700 comments this would mean more >> than >> almost 1500 queries to generate the page. >> >> I've modified this code extensively and now we use one query to fetch all >> the user details of all the people involved in posting. One query is >> used to >> fetch all the comments that have no parent. One query to fetch all the >> comments to do have parents. And if needed, one query to fetch the >> parent. >> All this data is then turned into one big nested array, which is >> passed by >> reference to the functions that actually print the data. Depending on how >> many comments there are this could result in a speed improvement of about >> 0% - 1000%. As you can imagine if you only have about 10 comments it >> would >> not mean much, with 500 comments it would reduce the amount of queries >> needed by about a 1000. It's a very big improvement. >> >> One other problem I've identified is table locking of the story table. >> The >> statistics are stored in the same table as the actual content of the >> story. >> So each time a story is displayed, it will use an UPDATE query and a >> SELECT >> query on the same table. With a lot of requests the table is constantly >> locked by the UPDATE queries and the SELECT queries are waiting. We've >> disabled the statistics for now, but we are investigating the >> possibility of >> moving the statistics to a separate table. >> >> Next is the database layer. The mysql_fetch_array() function has two >> arguments. The second determines what the function returns. Either an >> associative array, a numbered array or both. By default the function >> returns >> both. This is what Geeklog does. In most of the code only the associative >> array is used. Only in a couple of small instances the code requires an >> numbered array. What we have done is to instruct the mysql_fetch_array() >> function to return only an associative array by default. Only when the >> code >> requires a numbered array we request both. This should lower the >> amount of >> memory needed by Geeklog. >> >> The SEC_getUserGroups() function is also quite expensive. It is called >> throughout the generation a page and it does not cache the >> information. We' >> ve added a simple cache for the data that is fetched from the database >> which >> eliminates another 30 or so queries. >> >> Next is the index page. The COM_featuredCheck() function is executed >> every >> time the frontpage is requested. I've changed the loop that actually >> displays the stories on the frontpage and included a check to see if >> there >> is more than one featured story. If there is, the second story is not >> displayed as such and the featuredCheck() function is called. This again >> saves a couple of queries and the end result is the same. >> >> We are also using the mycal extension which I've almost completely >> rewritten. Mycal uses a query for every day that is displayed and >> after my >> modifications it only uses one query. A 27-34 reduction in queries. >> >> Now back to my first paragraph. I was pretty impressed with how easy >> it was >> to get used to the way everything works in Geeklog. It was pretty easy to >> understand and it looks like it was designed pretty well. But I was also >> horrified when I saw the enormous amount of queries that are used, but I >> guess Geeklog wasn't really designed with this kind of traffic and these >> enormous amounts of comments in mind. >> >> Most of the changes we've made are not yet running on the production >> server. >> Once we've properly tested everything and everything is stable, I'm >> willing >> to look at how we can give these changes back to Geeklog. As simple patch >> between the current version of Geeklog and Groklaw will be difficult, >> because we are using 1.3.8-1sr4 and it also includes a lot of Groklaw >> specific modifications. If you are interested in these modifications, >> please >> let me know and we'll work something out. >> >> If you want to talk to me about this you can e-mail. In addition to this >> I'll try to visit #geeklog as often as I can. >> >> Niels Leenheer >> -- project manager phpAdsNew >> >> >> > > _______________________________________________ > geeklog-devel mailing list > geeklog-devel at lists.geeklog.net > http://lists.geeklog.net/listinfo/geeklog-devel > From tony at tonybibbs.com Fri Mar 12 11:17:38 2004 From: tony at tonybibbs.com (Tony Bibbs) Date: Fri, 12 Mar 2004 10:17:38 -0600 Subject: [geeklog-devel] [Fwd: Re: Geeklog optimalisations] In-Reply-To: <4051D919.10802@abtech.org> References: <4051CBDE.4010407@tonybibbs.com> <4051D919.10802@abtech.org> Message-ID: <4051E2A2.6020605@tonybibbs.com> Yeah, I think we should start only with DB optimizations (SQL tuning, reducing queries, etc). I also think switching to InnoDB should be seriously considered. Doing so is something we can do relatively painlessly via upgrade scripts. I say start small, go after the bigger problems. Starting with comments would be good, however, I think someone should put together a plan on how to make sure during our performance analysis we cover all of the system . I still say leave the template stuff for 2.0...I don't think filesystem access is causing that much a of a problem. And yes, I need to get serious about 2.0...latley I have been telling myself to shit or get off the pot... --Tony Vincent Furia wrote: > I'm in! Though if we're going to get serious about upgrading geeklog > 1.3.x to reduce queries and seriously improve performance for high > traffic sites it might be time to start thinking about branching 1.3.x > off and using the main branch to start implementing these performance > improvements (and perhaps calling it 1.4.x). > > I actually don't think we're too far away from getting Geeklog to decent > speeds. As Niels pointed out (and as has been pointed out before I > think) we only really have one table locking issue. The database > queries are something that can be fixed with some intelligent caching > and better shaped queries. > > If we're feeling really brave, swithing template systems to a > pre-compiled template system like Smarty wouldn't be a bad idea. I > think that will do ALOT to improve performance. > > -Vinny > > Tony Bibbs wrote: > >> This is an FYI. We'll be discussing this on the development lists >> over the next few days (I hope). It's important we help Groklaw as >> best we can as they are one of our bigger sites and by them pushing >> the limits of Geeklog we can address their issues and make Geeklog a >> better product at the same time. >> >> --Tony >> >> --------------------- >> >> Niels, >> >> I think a bit of background is in order before you can understand how >> Geeklog got where it is. First, nearly all the code you are referring >> to is legacy code. It was there before I managed the project and it is >> still there under Dirk's management. In it's infancy, Geeklog was only >> servicing smaller sites so performance was never really an issue and, >> frankly, I was a bit young and dumb when I first got started with >> Geeklog so performance tuning PHP scripts wasn't even a consideration >> and my focus was on the feature set. >> >> Under Dirk's management, the feature set has continued to grow to the >> point that we have a large userbase and what you are encountering with >> Geeklog is only natural. Groklaw is clearly one of the biggest sites to >> run Geeklog. I have posted questions to our mailing lists asking about >> performance issues realted to bigger Geeklog sites getting no responses >> back so your email was a pleasant surprise. >> >> The long and the short of it is Geeklog has matured to a point where >> bigger sites are using it and we pushing the performance limits it has. >> Geeklog's database interaction has always been an issue for me and is >> a large part why I have chosen to get a new codebase up (i.e. Geeklog 2) >> while the 1.3.x continues. You are right, we need to address the >> performance issues and given the amount of work you have put into >> troubleshooting Groklaw I think you can play a critical part in that. >> >> What I would like to do is see us work closely with you to begin >> addressing these issues. A starting point would be to have a place >> where we can install a development version of Groklaw's database >> somewhere where we can run tests. Dirk and I don't have access to a >> database of that size and while we could fudge together some data using >> a real world example would sure be nice. Once we have a test bed, I'd >> be open to suggestions on how we might work on this to resolve your >> immediate issues *and* begin addressing performance tuning as a whole. >> >> #geeklog is where I dwell (though not always at the keyboard). If >> possible I'd like to see us discuss this on geeklog-devtalk. Niels, if >> you could join that list at http://lists.geeklog.net/listinfo we can >> carry this on there. In the meantime if you can catch Dirk or myself in >> IRC feel free to do so. FYI I'm out of town this weekend (FWIW I'm GMT >> -6) so I may not seem too responsive until I get back on Sunday. >> >> Thanks for contacting us, I'm sure we can address these issues. >> >> --Tony >> >> >> >> Niels Leenheer wrote: >> >>> Hi guys, >>> >>> First of all. What were you guys thinking? Sorry to be so rude, but I >>> simply >>> had to get that off my chest. I feel better now. I'm okay. Really. >>> >>> As some of you may be aware of Groklaw is using Geeklog. It has >>> turned in to >>> quite a busy website and stories with more than 700 comments are not >>> out of >>> the ordinary. In addition to this being slashdotted has become >>> normal. This >>> is where the problems started. The server can't handle much more. On >>> busy >>> days the website turns into a crawling slow pile of .. >>> >>> As a regular reader and volunteer of Groklaw I offered to take a look >>> at the >>> Geeklog source code and try to find some places that could benefit from >>> optimalisation. After some testing I've noticed that most of the >>> problems >>> are due to load on the database server. >>> >>> The first thing I started working on is the code that generates all the >>> comments. It turns out that for every comment at least two queries are >>> executed. For a story with more than 700 comments this would mean >>> more than >>> almost 1500 queries to generate the page. >>> >>> I've modified this code extensively and now we use one query to fetch >>> all >>> the user details of all the people involved in posting. One query is >>> used to >>> fetch all the comments that have no parent. One query to fetch all the >>> comments to do have parents. And if needed, one query to fetch the >>> parent. >>> All this data is then turned into one big nested array, which is >>> passed by >>> reference to the functions that actually print the data. Depending on >>> how >>> many comments there are this could result in a speed improvement of >>> about >>> 0% - 1000%. As you can imagine if you only have about 10 comments it >>> would >>> not mean much, with 500 comments it would reduce the amount of queries >>> needed by about a 1000. It's a very big improvement. >>> >>> One other problem I've identified is table locking of the story >>> table. The >>> statistics are stored in the same table as the actual content of the >>> story. >>> So each time a story is displayed, it will use an UPDATE query and a >>> SELECT >>> query on the same table. With a lot of requests the table is constantly >>> locked by the UPDATE queries and the SELECT queries are waiting. We've >>> disabled the statistics for now, but we are investigating the >>> possibility of >>> moving the statistics to a separate table. >>> >>> Next is the database layer. The mysql_fetch_array() function has two >>> arguments. The second determines what the function returns. Either an >>> associative array, a numbered array or both. By default the function >>> returns >>> both. This is what Geeklog does. In most of the code only the >>> associative >>> array is used. Only in a couple of small instances the code requires an >>> numbered array. What we have done is to instruct the mysql_fetch_array() >>> function to return only an associative array by default. Only when >>> the code >>> requires a numbered array we request both. This should lower the >>> amount of >>> memory needed by Geeklog. >>> >>> The SEC_getUserGroups() function is also quite expensive. It is called >>> throughout the generation a page and it does not cache the >>> information. We' >>> ve added a simple cache for the data that is fetched from the >>> database which >>> eliminates another 30 or so queries. >>> >>> Next is the index page. The COM_featuredCheck() function is executed >>> every >>> time the frontpage is requested. I've changed the loop that actually >>> displays the stories on the frontpage and included a check to see if >>> there >>> is more than one featured story. If there is, the second story is not >>> displayed as such and the featuredCheck() function is called. This again >>> saves a couple of queries and the end result is the same. >>> >>> We are also using the mycal extension which I've almost completely >>> rewritten. Mycal uses a query for every day that is displayed and >>> after my >>> modifications it only uses one query. A 27-34 reduction in queries. >>> >>> Now back to my first paragraph. I was pretty impressed with how easy >>> it was >>> to get used to the way everything works in Geeklog. It was pretty >>> easy to >>> understand and it looks like it was designed pretty well. But I was also >>> horrified when I saw the enormous amount of queries that are used, but I >>> guess Geeklog wasn't really designed with this kind of traffic and these >>> enormous amounts of comments in mind. >>> >>> Most of the changes we've made are not yet running on the production >>> server. >>> Once we've properly tested everything and everything is stable, I'm >>> willing >>> to look at how we can give these changes back to Geeklog. As simple >>> patch >>> between the current version of Geeklog and Groklaw will be difficult, >>> because we are using 1.3.8-1sr4 and it also includes a lot of Groklaw >>> specific modifications. If you are interested in these modifications, >>> please >>> let me know and we'll work something out. >>> >>> If you want to talk to me about this you can e-mail. In addition to this >>> I'll try to visit #geeklog as often as I can. >>> >>> Niels Leenheer >>> -- project manager phpAdsNew >>> >>> >>> >> >> _______________________________________________ >> geeklog-devel mailing list >> geeklog-devel at lists.geeklog.net >> http://lists.geeklog.net/listinfo/geeklog-devel >> > _______________________________________________ > geeklog-devel mailing list > geeklog-devel at lists.geeklog.net > http://lists.geeklog.net/listinfo/geeklog-devel From geeklog at langfamily.ca Fri Mar 12 11:30:40 2004 From: geeklog at langfamily.ca (Blaine Lang) Date: Fri, 12 Mar 2004 11:30:40 -0500 Subject: [geeklog-devel] [Fwd: Re: Geeklog optimalisations] References: <4051CBDE.4010407@tonybibbs.com> <4051D919.10802@abtech.org> <4051E2A2.6020605@tonybibbs.com> Message-ID: <006f01c4084f$5b593cb0$650a10ac@XPBL2> I have to agree with that Geeklog as it exists is fine for 98% of our users but addressing some of these issues can only make it better. We just need to be careful as a lot of the bugs that we have been addressing are permission based ones and we don't want to introduce more issues that will effect the 98% of our users. I know it is well known, that adding new code and features often introduces new bugs so need to keep that in mind when assessing the effort for such a project. What about using PHP Sessions to cache some of the common information and eliminate a lot of the SQL queries. Possible areas would be: - User access rights and group membership - $_USER array It would be nice for Plugins to have access to SESSIONS as well. ----- Original Message ----- From: "Tony Bibbs" To: Sent: Friday, March 12, 2004 11:17 AM Subject: Re: [geeklog-devel] [Fwd: Re: Geeklog optimalisations] > Yeah, I think we should start only with DB optimizations (SQL tuning, > reducing queries, etc). I also think switching to InnoDB should be > seriously considered. Doing so is something we can do relatively > painlessly via upgrade scripts. I say start small, go after the bigger > problems. Starting with comments would be good, however, I think > someone should put together a plan on how to make sure during our > performance analysis we cover all of the system . > > I still say leave the template stuff for 2.0...I don't think filesystem > access is causing that much a of a problem. And yes, I need to get > serious about 2.0...latley I have been telling myself to shit or get off > the pot... > > --Tony > > Vincent Furia wrote: > > I'm in! Though if we're going to get serious about upgrading geeklog > > 1.3.x to reduce queries and seriously improve performance for high > > traffic sites it might be time to start thinking about branching 1.3.x > > off and using the main branch to start implementing these performance > > improvements (and perhaps calling it 1.4.x). > > > > I actually don't think we're too far away from getting Geeklog to decent > > speeds. As Niels pointed out (and as has been pointed out before I > > think) we only really have one table locking issue. The database > > queries are something that can be fixed with some intelligent caching > > and better shaped queries. > > > > If we're feeling really brave, swithing template systems to a > > pre-compiled template system like Smarty wouldn't be a bad idea. I > > think that will do ALOT to improve performance. > > > > -Vinny > > > > Tony Bibbs wrote: > > > >> This is an FYI. We'll be discussing this on the development lists > >> over the next few days (I hope). It's important we help Groklaw as > >> best we can as they are one of our bigger sites and by them pushing > >> the limits of Geeklog we can address their issues and make Geeklog a > >> better product at the same time. > >> > >> --Tony > >> > >> --------------------- > >> > >> Niels, > >> > >> I think a bit of background is in order before you can understand how > >> Geeklog got where it is. First, nearly all the code you are referring > >> to is legacy code. It was there before I managed the project and it is > >> still there under Dirk's management. In it's infancy, Geeklog was only > >> servicing smaller sites so performance was never really an issue and, > >> frankly, I was a bit young and dumb when I first got started with > >> Geeklog so performance tuning PHP scripts wasn't even a consideration > >> and my focus was on the feature set. > >> > >> Under Dirk's management, the feature set has continued to grow to the > >> point that we have a large userbase and what you are encountering with > >> Geeklog is only natural. Groklaw is clearly one of the biggest sites to > >> run Geeklog. I have posted questions to our mailing lists asking about > >> performance issues realted to bigger Geeklog sites getting no responses > >> back so your email was a pleasant surprise. > >> > >> The long and the short of it is Geeklog has matured to a point where > >> bigger sites are using it and we pushing the performance limits it has. > >> Geeklog's database interaction has always been an issue for me and is > >> a large part why I have chosen to get a new codebase up (i.e. Geeklog 2) > >> while the 1.3.x continues. You are right, we need to address the > >> performance issues and given the amount of work you have put into > >> troubleshooting Groklaw I think you can play a critical part in that. > >> > >> What I would like to do is see us work closely with you to begin > >> addressing these issues. A starting point would be to have a place > >> where we can install a development version of Groklaw's database > >> somewhere where we can run tests. Dirk and I don't have access to a > >> database of that size and while we could fudge together some data using > >> a real world example would sure be nice. Once we have a test bed, I'd > >> be open to suggestions on how we might work on this to resolve your > >> immediate issues *and* begin addressing performance tuning as a whole. > >> > >> #geeklog is where I dwell (though not always at the keyboard). If > >> possible I'd like to see us discuss this on geeklog-devtalk. Niels, if > >> you could join that list at http://lists.geeklog.net/listinfo we can > >> carry this on there. In the meantime if you can catch Dirk or myself in > >> IRC feel free to do so. FYI I'm out of town this weekend (FWIW I'm GMT > >> -6) so I may not seem too responsive until I get back on Sunday. > >> > >> Thanks for contacting us, I'm sure we can address these issues. > >> > >> --Tony > >> > >> > >> > >> Niels Leenheer wrote: > >> > >>> Hi guys, > >>> > >>> First of all. What were you guys thinking? Sorry to be so rude, but I > >>> simply > >>> had to get that off my chest. I feel better now. I'm okay. Really. > >>> > >>> As some of you may be aware of Groklaw is using Geeklog. It has > >>> turned in to > >>> quite a busy website and stories with more than 700 comments are not > >>> out of > >>> the ordinary. In addition to this being slashdotted has become > >>> normal. This > >>> is where the problems started. The server can't handle much more. On > >>> busy > >>> days the website turns into a crawling slow pile of .. > >>> > >>> As a regular reader and volunteer of Groklaw I offered to take a look > >>> at the > >>> Geeklog source code and try to find some places that could benefit from > >>> optimalisation. After some testing I've noticed that most of the > >>> problems > >>> are due to load on the database server. > >>> > >>> The first thing I started working on is the code that generates all the > >>> comments. It turns out that for every comment at least two queries are > >>> executed. For a story with more than 700 comments this would mean > >>> more than > >>> almost 1500 queries to generate the page. > >>> > >>> I've modified this code extensively and now we use one query to fetch > >>> all > >>> the user details of all the people involved in posting. One query is > >>> used to > >>> fetch all the comments that have no parent. One query to fetch all the > >>> comments to do have parents. And if needed, one query to fetch the > >>> parent. > >>> All this data is then turned into one big nested array, which is > >>> passed by > >>> reference to the functions that actually print the data. Depending on > >>> how > >>> many comments there are this could result in a speed improvement of > >>> about > >>> 0% - 1000%. As you can imagine if you only have about 10 comments it > >>> would > >>> not mean much, with 500 comments it would reduce the amount of queries > >>> needed by about a 1000. It's a very big improvement. > >>> > >>> One other problem I've identified is table locking of the story > >>> table. The > >>> statistics are stored in the same table as the actual content of the > >>> story. > >>> So each time a story is displayed, it will use an UPDATE query and a > >>> SELECT > >>> query on the same table. With a lot of requests the table is constantly > >>> locked by the UPDATE queries and the SELECT queries are waiting. We've > >>> disabled the statistics for now, but we are investigating the > >>> possibility of > >>> moving the statistics to a separate table. > >>> > >>> Next is the database layer. The mysql_fetch_array() function has two > >>> arguments. The second determines what the function returns. Either an > >>> associative array, a numbered array or both. By default the function > >>> returns > >>> both. This is what Geeklog does. In most of the code only the > >>> associative > >>> array is used. Only in a couple of small instances the code requires an > >>> numbered array. What we have done is to instruct the mysql_fetch_array() > >>> function to return only an associative array by default. Only when > >>> the code > >>> requires a numbered array we request both. This should lower the > >>> amount of > >>> memory needed by Geeklog. > >>> > >>> The SEC_getUserGroups() function is also quite expensive. It is called > >>> throughout the generation a page and it does not cache the > >>> information. We' > >>> ve added a simple cache for the data that is fetched from the > >>> database which > >>> eliminates another 30 or so queries. > >>> > >>> Next is the index page. The COM_featuredCheck() function is executed > >>> every > >>> time the frontpage is requested. I've changed the loop that actually > >>> displays the stories on the frontpage and included a check to see if > >>> there > >>> is more than one featured story. If there is, the second story is not > >>> displayed as such and the featuredCheck() function is called. This again > >>> saves a couple of queries and the end result is the same. > >>> > >>> We are also using the mycal extension which I've almost completely > >>> rewritten. Mycal uses a query for every day that is displayed and > >>> after my > >>> modifications it only uses one query. A 27-34 reduction in queries. > >>> > >>> Now back to my first paragraph. I was pretty impressed with how easy > >>> it was > >>> to get used to the way everything works in Geeklog. It was pretty > >>> easy to > >>> understand and it looks like it was designed pretty well. But I was also > >>> horrified when I saw the enormous amount of queries that are used, but I > >>> guess Geeklog wasn't really designed with this kind of traffic and these > >>> enormous amounts of comments in mind. > >>> > >>> Most of the changes we've made are not yet running on the production > >>> server. > >>> Once we've properly tested everything and everything is stable, I'm > >>> willing > >>> to look at how we can give these changes back to Geeklog. As simple > >>> patch > >>> between the current version of Geeklog and Groklaw will be difficult, > >>> because we are using 1.3.8-1sr4 and it also includes a lot of Groklaw > >>> specific modifications. If you are interested in these modifications, > >>> please > >>> let me know and we'll work something out. > >>> > >>> If you want to talk to me about this you can e-mail. In addition to this > >>> I'll try to visit #geeklog as often as I can. > >>> > >>> Niels Leenheer > >>> -- project manager phpAdsNew > >>> > >>> > >>> > >> > >> _______________________________________________ > >> geeklog-devel mailing list > >> geeklog-devel at lists.geeklog.net > >> http://lists.geeklog.net/listinfo/geeklog-devel > >> > > _______________________________________________ > > geeklog-devel mailing list > > geeklog-devel at lists.geeklog.net > > http://lists.geeklog.net/listinfo/geeklog-devel > _______________________________________________ > geeklog-devel mailing list > geeklog-devel at lists.geeklog.net > http://lists.geeklog.net/listinfo/geeklog-devel From tony at tonybibbs.com Fri Mar 12 11:44:58 2004 From: tony at tonybibbs.com (Tony Bibbs) Date: Fri, 12 Mar 2004 10:44:58 -0600 Subject: [geeklog-devel] [Fwd: Re: Geeklog optimalisations] In-Reply-To: <006f01c4084f$5b593cb0$650a10ac@XPBL2> References: <4051CBDE.4010407@tonybibbs.com> <4051D919.10802@abtech.org> <4051E2A2.6020605@tonybibbs.com> <006f01c4084f$5b593cb0$650a10ac@XPBL2> Message-ID: <4051E90A.7070308@tonybibbs.com> I agree session management should be considered but, again, this would be a lot of work. Having a user object in the session would be a tremendous help. Switching to PHP4 sessions might be too much work (research is needed) but you may be able to emulate PHP4 sessions by adding a data field to the current sessions table and then do serialization/deserialization from that. I digress. The big issue at hand is addressing a few short term performance problem. I think looking into Niel's customizations is a starting point. Blaine is right, we need a fix that won't affect the majority of users yet give sites like Groklaw and our big Mac-related sites more bang for their buck. I think addressing the comments is a big issue. particularly considering the differ modes you can view the comments in. --Tony Blaine Lang wrote: > I have to agree with that Geeklog as it exists is fine for 98% of our users > but addressing some of these issues can only make it better. > > We just need to be careful as a lot of the bugs that we have been addressing > are permission based ones and we don't want to introduce more issues that > will effect the 98% of our users. I know it is well known, that adding new > code and features often introduces new bugs so need to keep that in mind > when assessing the effort for such a project. > > What about using PHP Sessions to cache some of the common information and > eliminate a lot of the SQL queries. > Possible areas would be: > - User access rights and group membership > - $_USER array > > It would be nice for Plugins to have access to SESSIONS as well. > > ----- Original Message ----- > From: "Tony Bibbs" > To: > Sent: Friday, March 12, 2004 11:17 AM > Subject: Re: [geeklog-devel] [Fwd: Re: Geeklog optimalisations] > > > >>Yeah, I think we should start only with DB optimizations (SQL tuning, >>reducing queries, etc). I also think switching to InnoDB should be >>seriously considered. Doing so is something we can do relatively >>painlessly via upgrade scripts. I say start small, go after the bigger >>problems. Starting with comments would be good, however, I think >>someone should put together a plan on how to make sure during our >>performance analysis we cover all of the system . >> >>I still say leave the template stuff for 2.0...I don't think filesystem >>access is causing that much a of a problem. And yes, I need to get >>serious about 2.0...latley I have been telling myself to shit or get off >> the pot... >> >>--Tony >> >>Vincent Furia wrote: >> >>>I'm in! Though if we're going to get serious about upgrading geeklog >>>1.3.x to reduce queries and seriously improve performance for high >>>traffic sites it might be time to start thinking about branching 1.3.x >>>off and using the main branch to start implementing these performance >>>improvements (and perhaps calling it 1.4.x). >>> >>>I actually don't think we're too far away from getting Geeklog to decent >>>speeds. As Niels pointed out (and as has been pointed out before I >>>think) we only really have one table locking issue. The database >>>queries are something that can be fixed with some intelligent caching >>>and better shaped queries. >>> >>>If we're feeling really brave, swithing template systems to a >>>pre-compiled template system like Smarty wouldn't be a bad idea. I >>>think that will do ALOT to improve performance. >>> >>>-Vinny >>> >>>Tony Bibbs wrote: >>> >>> >>>>This is an FYI. We'll be discussing this on the development lists >>>>over the next few days (I hope). It's important we help Groklaw as >>>>best we can as they are one of our bigger sites and by them pushing >>>>the limits of Geeklog we can address their issues and make Geeklog a >>>>better product at the same time. >>>> >>>>--Tony >>>> >>>>--------------------- >>>> >>>>Niels, >>>> >>>>I think a bit of background is in order before you can understand how >>>>Geeklog got where it is. First, nearly all the code you are referring >>>>to is legacy code. It was there before I managed the project and it is >>>>still there under Dirk's management. In it's infancy, Geeklog was only >>>>servicing smaller sites so performance was never really an issue and, >>>>frankly, I was a bit young and dumb when I first got started with >>>>Geeklog so performance tuning PHP scripts wasn't even a consideration >>>>and my focus was on the feature set. >>>> >>>>Under Dirk's management, the feature set has continued to grow to the >>>>point that we have a large userbase and what you are encountering with >>>>Geeklog is only natural. Groklaw is clearly one of the biggest sites > > to > >>>>run Geeklog. I have posted questions to our mailing lists asking about >>>>performance issues realted to bigger Geeklog sites getting no responses >>>>back so your email was a pleasant surprise. >>>> >>>>The long and the short of it is Geeklog has matured to a point where >>>>bigger sites are using it and we pushing the performance limits it has. >>>> Geeklog's database interaction has always been an issue for me and is >>>>a large part why I have chosen to get a new codebase up (i.e. Geeklog > > 2) > >>>>while the 1.3.x continues. You are right, we need to address the >>>>performance issues and given the amount of work you have put into >>>>troubleshooting Groklaw I think you can play a critical part in that. >>>> >>>>What I would like to do is see us work closely with you to begin >>>>addressing these issues. A starting point would be to have a place >>>>where we can install a development version of Groklaw's database >>>>somewhere where we can run tests. Dirk and I don't have access to a >>>>database of that size and while we could fudge together some data using >>>>a real world example would sure be nice. Once we have a test bed, I'd >>>>be open to suggestions on how we might work on this to resolve your >>>>immediate issues *and* begin addressing performance tuning as a whole. >>>> >>>>#geeklog is where I dwell (though not always at the keyboard). If >>>>possible I'd like to see us discuss this on geeklog-devtalk. Niels, if >>>>you could join that list at http://lists.geeklog.net/listinfo we can >>>>carry this on there. In the meantime if you can catch Dirk or myself > > in > >>>>IRC feel free to do so. FYI I'm out of town this weekend (FWIW I'm GMT >>>>-6) so I may not seem too responsive until I get back on Sunday. >>>> >>>>Thanks for contacting us, I'm sure we can address these issues. >>>> >>>>--Tony >>>> >>>> >>>> >>>>Niels Leenheer wrote: >>>> >>>> >>>>>Hi guys, >>>>> >>>>>First of all. What were you guys thinking? Sorry to be so rude, but I >>>>>simply >>>>>had to get that off my chest. I feel better now. I'm okay. Really. >>>>> >>>>>As some of you may be aware of Groklaw is using Geeklog. It has >>>>>turned in to >>>>>quite a busy website and stories with more than 700 comments are not >>>>>out of >>>>>the ordinary. In addition to this being slashdotted has become >>>>>normal. This >>>>>is where the problems started. The server can't handle much more. On >>>>>busy >>>>>days the website turns into a crawling slow pile of .. >>>>> >>>>>As a regular reader and volunteer of Groklaw I offered to take a look >>>>>at the >>>>>Geeklog source code and try to find some places that could benefit > > from > >>>>>optimalisation. After some testing I've noticed that most of the >>>>>problems >>>>>are due to load on the database server. >>>>> >>>>>The first thing I started working on is the code that generates all > > the > >>>>>comments. It turns out that for every comment at least two queries are >>>>>executed. For a story with more than 700 comments this would mean >>>>>more than >>>>>almost 1500 queries to generate the page. >>>>> >>>>>I've modified this code extensively and now we use one query to fetch >>>>>all >>>>>the user details of all the people involved in posting. One query is >>>>>used to >>>>>fetch all the comments that have no parent. One query to fetch all the >>>>>comments to do have parents. And if needed, one query to fetch the >>>>>parent. >>>>>All this data is then turned into one big nested array, which is >>>>>passed by >>>>>reference to the functions that actually print the data. Depending on >>>>>how >>>>>many comments there are this could result in a speed improvement of >>>>>about >>>>>0% - 1000%. As you can imagine if you only have about 10 comments it >>>>>would >>>>>not mean much, with 500 comments it would reduce the amount of queries >>>>>needed by about a 1000. It's a very big improvement. >>>>> >>>>>One other problem I've identified is table locking of the story >>>>>table. The >>>>>statistics are stored in the same table as the actual content of the >>>>>story. >>>>>So each time a story is displayed, it will use an UPDATE query and a >>>>>SELECT >>>>>query on the same table. With a lot of requests the table is > > constantly > >>>>>locked by the UPDATE queries and the SELECT queries are waiting. We've >>>>>disabled the statistics for now, but we are investigating the >>>>>possibility of >>>>>moving the statistics to a separate table. >>>>> >>>>>Next is the database layer. The mysql_fetch_array() function has two >>>>>arguments. The second determines what the function returns. Either an >>>>>associative array, a numbered array or both. By default the function >>>>>returns >>>>>both. This is what Geeklog does. In most of the code only the >>>>>associative >>>>>array is used. Only in a couple of small instances the code requires > > an > >>>>>numbered array. What we have done is to instruct the > > mysql_fetch_array() > >>>>>function to return only an associative array by default. Only when >>>>>the code >>>>>requires a numbered array we request both. This should lower the >>>>>amount of >>>>>memory needed by Geeklog. >>>>> >>>>>The SEC_getUserGroups() function is also quite expensive. It is called >>>>>throughout the generation a page and it does not cache the >>>>>information. We' >>>>>ve added a simple cache for the data that is fetched from the >>>>>database which >>>>>eliminates another 30 or so queries. >>>>> >>>>>Next is the index page. The COM_featuredCheck() function is executed >>>>>every >>>>>time the frontpage is requested. I've changed the loop that actually >>>>>displays the stories on the frontpage and included a check to see if >>>>>there >>>>>is more than one featured story. If there is, the second story is not >>>>>displayed as such and the featuredCheck() function is called. This > > again > >>>>>saves a couple of queries and the end result is the same. >>>>> >>>>>We are also using the mycal extension which I've almost completely >>>>>rewritten. Mycal uses a query for every day that is displayed and >>>>>after my >>>>>modifications it only uses one query. A 27-34 reduction in queries. >>>>> >>>>>Now back to my first paragraph. I was pretty impressed with how easy >>>>>it was >>>>>to get used to the way everything works in Geeklog. It was pretty >>>>>easy to >>>>>understand and it looks like it was designed pretty well. But I was > > also > >>>>>horrified when I saw the enormous amount of queries that are used, but > > I > >>>>>guess Geeklog wasn't really designed with this kind of traffic and > > these > >>>>>enormous amounts of comments in mind. >>>>> >>>>>Most of the changes we've made are not yet running on the production >>>>>server. >>>>>Once we've properly tested everything and everything is stable, I'm >>>>>willing >>>>>to look at how we can give these changes back to Geeklog. As simple >>>>>patch >>>>>between the current version of Geeklog and Groklaw will be difficult, >>>>>because we are using 1.3.8-1sr4 and it also includes a lot of Groklaw >>>>>specific modifications. If you are interested in these modifications, >>>>>please >>>>>let me know and we'll work something out. >>>>> >>>>>If you want to talk to me about this you can e-mail. In addition to > > this > >>>>>I'll try to visit #geeklog as often as I can. >>>>> >>>>>Niels Leenheer >>>>>-- project manager phpAdsNew >>>>> >>>>> >>>>> >>>> >>>>_______________________________________________ >>>>geeklog-devel mailing list >>>>geeklog-devel at lists.geeklog.net >>>>http://lists.geeklog.net/listinfo/geeklog-devel >>>> >>> >>>_______________________________________________ >>>geeklog-devel mailing list >>>geeklog-devel at lists.geeklog.net >>>http://lists.geeklog.net/listinfo/geeklog-devel >> >>_______________________________________________ >>geeklog-devel mailing list >>geeklog-devel at lists.geeklog.net >>http://lists.geeklog.net/listinfo/geeklog-devel > > > _______________________________________________ > geeklog-devel mailing list > geeklog-devel at lists.geeklog.net > http://lists.geeklog.net/listinfo/geeklog-devel From tony at tonybibbs.com Mon Mar 15 13:54:01 2004 From: tony at tonybibbs.com (Tony Bibbs) Date: Mon, 15 Mar 2004 12:54:01 -0600 Subject: [geeklog-devel] Better way to manage themes stuff Message-ID: <4055FBC9.2000908@tonybibbs.com> Hey, I was talking with a coworker doing a few GL installs for some personal things he has (including one for his church) anyway, he ased a question around knowing what template variables are available to a page. That got me to thinking a bit about the translation stuff for GL2. Right now there is a PHP shell script that can recursively parse .php pages calls to the getText() function. With a little modification it would be easy enough to have a hacked version of that script pull out calls to setVariable(). In addition to that we can have it look at which actual template it is working on and then print a file that lists the template file name (absolute path) and variables used. Now that by itself doesn't do a whole lot but maybe that could be the basis of what gets used for documentation, no? Heck, even knowing what variables are available in each template without descriptions would help since many of the variables have descriptive names. --Tony From geeklog at langfamily.ca Tue Mar 16 09:07:35 2004 From: geeklog at langfamily.ca (Blaine Lang) Date: Tue, 16 Mar 2004 09:07:35 -0500 Subject: [geeklog-devel] Better way to manage themes stuff References: <4055FBC9.2000908@tonybibbs.com> Message-ID: <004301c40b60$081498b0$650a10ac@XPBL2> Tony, I think this would be very helpfull. Blaine ----- Original Message ----- From: "Tony Bibbs" To: "Geeklog" Sent: Monday, March 15, 2004 1:54 PM Subject: [geeklog-devel] Better way to manage themes stuff > Hey, I was talking with a coworker doing a few GL installs for some > personal things he has (including one for his church) anyway, he ased a > question around knowing what template variables are available to a page. > That got me to thinking a bit about the translation stuff for GL2. > Right now there is a PHP shell script that can recursively parse .php > pages calls to the getText() function. With a little modification it > would be easy enough to have a hacked version of that script pull out > calls to setVariable(). In addition to that we can have it look at > which actual template it is working on and then print a file that lists > the template file name (absolute path) and variables used. > > Now that by itself doesn't do a whole lot but maybe that could be the > basis of what gets used for documentation, no? Heck, even knowing what > variables are available in each template without descriptions would help > since many of the variables have descriptive names. > > --Tony > _______________________________________________ > geeklog-devel mailing list > geeklog-devel at lists.geeklog.net > http://lists.geeklog.net/listinfo/geeklog-devel From tony at tonybibbs.com Tue Mar 16 10:02:07 2004 From: tony at tonybibbs.com (Tony Bibbs) Date: Tue, 16 Mar 2004 09:02:07 -0600 Subject: [geeklog-devel] Better way to manage themes stuff In-Reply-To: <004301c40b60$081498b0$650a10ac@XPBL2> References: <4055FBC9.2000908@tonybibbs.com> <004301c40b60$081498b0$650a10ac@XPBL2> Message-ID: <405716EF.4000808@tonybibbs.com> Yeah, I'll look into this. I'm hoping this isn't something that will take much time. --Tony Blaine Lang wrote: > Tony, > > I think this would be very helpfull. > > Blaine > ----- Original Message ----- > From: "Tony Bibbs" > To: "Geeklog" > Sent: Monday, March 15, 2004 1:54 PM > Subject: [geeklog-devel] Better way to manage themes stuff > > > >>Hey, I was talking with a coworker doing a few GL installs for some >>personal things he has (including one for his church) anyway, he ased a >>question around knowing what template variables are available to a page. >> That got me to thinking a bit about the translation stuff for GL2. >>Right now there is a PHP shell script that can recursively parse .php >>pages calls to the getText() function. With a little modification it >>would be easy enough to have a hacked version of that script pull out >>calls to setVariable(). In addition to that we can have it look at >>which actual template it is working on and then print a file that lists >>the template file name (absolute path) and variables used. >> >>Now that by itself doesn't do a whole lot but maybe that could be the >>basis of what gets used for documentation, no? Heck, even knowing what >>variables are available in each template without descriptions would help >>since many of the variables have descriptive names. >> >>--Tony >>_______________________________________________ >>geeklog-devel mailing list >>geeklog-devel at lists.geeklog.net >>http://lists.geeklog.net/listinfo/geeklog-devel > > _______________________________________________ > geeklog-devel mailing list > geeklog-devel at lists.geeklog.net > http://lists.geeklog.net/listinfo/geeklog-devel From tony at tonybibbs.com Tue Mar 16 17:57:10 2004 From: tony at tonybibbs.com (Tony Bibbs) Date: Tue, 16 Mar 2004 16:57:10 -0600 Subject: [geeklog-devel] Request to create new library called lib-account.php Message-ID: <40578646.5010004@tonybibbs.com> Hi, I have been trying to use Geeklog here at work and the only way I will be able to do so is if I tie GL into our authentication system here (which uses SSO). At a 10,000ft view all I want to do is bypass Geeklog's login feature for our employees and use our system instead. For customers not using our auth system I want them to use the normal baked in account management features that Geeklog has. To be clear, all I want to do is bypass geeklog's login feature in some cases. Here is the scenario. One of our workers logs into Geeklog using our auth system. If the login on our side is good we issue an SSO token to the session cookie and return that SSO token to Geeklog. When geeklog gets the SSO token, I validate the token and if it is okay I check to see if the user has an GL account. If not I create one and them log them in by writing the necessary Geeklog cookies. If they do have an account I simply login them in. Our auth system works the same way that Passport, Project Liberty, etc does and the only way I could get this to work was to remove all the functions in public_html/users.php (except for the profile function) and put them in lib-acccount.php. In this manner I now have access to teh fucntions I need access to without compromising the way users.php expects to work. I'd like to recommend this minor change be put into Geeklog. I have attached the lib-account.php and a hacked version of users.php. My version of users.php still has all the old function stubs that call the ACCT_ equivalents in lib-account and I realize that the final version would have to change. Of more interest would be the login.php form I created that handles all this crap. Again, it is a hack but it works. Note there is a library included at teh top you don't have access to but you shoudl get the jist of what I needed. If we can agree on this, I will submit a new, polished version of users.php along with my lib-account.php to CVS but before I did so I wanted to open a dialog. --Tony -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An HTML attachment was scrubbed... URL: From dirk at haun-online.de Sun Mar 21 16:30:39 2004 From: dirk at haun-online.de (Dirk Haun) Date: Sun, 21 Mar 2004 22:30:39 +0100 Subject: [geeklog-devel] [Fwd: Re: Geeklog optimalisations] In-Reply-To: <4051CBDE.4010407@tonybibbs.com> References: <4051CBDE.4010407@tonybibbs.com> Message-ID: <20040321213039.32493@smtp.haun-online.de> Niels Leenheer wrote: >> Next is the database layer. The mysql_fetch_array() function has two >> arguments. The second determines what the function returns. Either an >> associative array, a numbered array or both. By default the function returns >> both. This is what Geeklog does. In most of the code only the associative >> array is used. Only in a couple of small instances the code requires an >> numbered array. What we have done is to instruct the mysql_fetch_array() >> function to return only an associative array by default. Only when the code >> requires a numbered array we request both. This should lower the amount of >> memory needed by Geeklog. This is now implemented in CVS. Per default, DB_fetchArray will only return the associative array indices. If you need the numeric ones as well, you'll have to pass a new (optional) second parameter: DB_fetchArray ($result, true); Thanks to Niels for helping me find all the places where the second parameter is needed. I've also fixed the first batch of bugs that have come up (from docs/ history - Tony, any news on CVS notifications?) - Removed extra quotes in SQL statements in admin/block.php to ensure compatibility with old MySQL versions. - Fixed SQL error in COM_showTopics when users tried to exclude topics in their preferences. - In admin/database.php, if the is_executable function is not available (e.g. on Windows), do at least a file_exists to check if the mysqldump executable exists in the path given in config.php. bye, Dirk -- http://www.haun-online.de/ http://geeklog.info/ From tony at tonybibbs.com Mon Mar 22 09:45:15 2004 From: tony at tonybibbs.com (Tony Bibbs) Date: Mon, 22 Mar 2004 08:45:15 -0600 Subject: [geeklog-devel] [Fwd: Re: Geeklog optimalisations] In-Reply-To: <20040321213039.32493@smtp.haun-online.de> References: <4051CBDE.4010407@tonybibbs.com> <20040321213039.32493@smtp.haun-online.de> Message-ID: <405EFBFB.4090509@tonybibbs.com> News on CVS notification is this. I'm only a week or so away from moving the site back to the primary server (which now has 1GB of RAM and two new 18.1GB drives to add to the 3 existing ones which will be RAID5'd). Instead of fixing the problem on the current server (which has been futile) I'm going to wait until I cut back over to the primary server (at the new ISP) and go from there. --Tony Dirk Haun wrote: > Niels Leenheer wrote: > > >>>Next is the database layer. The mysql_fetch_array() function has two >>>arguments. The second determines what the function returns. Either an >>>associative array, a numbered array or both. By default the function > > returns > >>>both. This is what Geeklog does. In most of the code only the associative >>>array is used. Only in a couple of small instances the code requires an >>>numbered array. What we have done is to instruct the mysql_fetch_array() >>>function to return only an associative array by default. Only when the code >>>requires a numbered array we request both. This should lower the amount of >>>memory needed by Geeklog. > > > This is now implemented in CVS. Per default, DB_fetchArray will only > return the associative array indices. If you need the numeric ones as > well, you'll have to pass a new (optional) second parameter: > DB_fetchArray ($result, true); > > Thanks to Niels for helping me find all the places where the second > parameter is needed. > > I've also fixed the first batch of bugs that have come up (from docs/ > history - Tony, any news on CVS notifications?) > > - Removed extra quotes in SQL statements in admin/block.php to ensure > compatibility with old MySQL versions. > - Fixed SQL error in COM_showTopics when users tried to exclude topics in > their > preferences. > - In admin/database.php, if the is_executable function is not available (e.g. > on Windows), do at least a file_exists to check if the mysqldump executable > exists in the path given in config.php. > > bye, Dirk > > From dirk at haun-online.de Mon Mar 22 14:13:07 2004 From: dirk at haun-online.de (Dirk Haun) Date: Mon, 22 Mar 2004 20:13:07 +0100 Subject: [geeklog-devel] Experimental InnoDB support Message-ID: <20040322191307.18127@smtp.haun-online.de> (k, so I'll keep on playing human gateway for the CVS changes ;-) I've changed the install script to check for a MySQL version that supports InnoDB tables. If that is the case, a fresh install will now use InnoDB instead of MyISAM tables. I'd like to hear some feedback on this. Also, if anyone is running an old version of MySQL (something around 3.23.34) I'd like to know if the check causes any problems there. bye, Dirk -- http://www.haun-online.de/ http://www.haun.info/ From geeklog at langfamily.ca Tue Mar 23 08:52:30 2004 From: geeklog at langfamily.ca (Blaine Lang) Date: Tue, 23 Mar 2004 08:52:30 -0500 Subject: [geeklog-devel] Downloader and Upload classes Message-ID: <002501c410de$15f776e0$650a10ac@XPBL2> I have a plugin that uses these two classes and used a plugin config variable to define the allowable filetypes that could be uploaded and downloaded. I wanted to use the same define to set the allowable upload and download filetypes in the class but the class uses two different array definition methods. The upload.class uses this method $this->_availableMimeTypes = array( 'application/pdf' => '.pdf', 'application/x-shockwave-flash' => '.swf', 'application/msword' => '.doc', 'application/vnd.ms-excel' => '.xls' ) The downloader.class uses the reverse method $this->_availableMimeTypes = array( 'tgz' => 'application/x-gzip-compressed', 'gz' => 'application/x-gzip-compressed', 'zip' => 'application/x-zip-compresseed', 'tar' => 'application/x-tar', 'php' => 'text/plain', 'phps' => 'text/plain' ) There may be a reason this was done this way -- or was the classes were developed at different times and thats just the way it is. Should we change one and align them to use the same method? Blaine -------------- next part -------------- An HTML attachment was scrubbed... URL: From geeklog at thehares.com Tue Mar 23 20:24:10 2004 From: geeklog at thehares.com (Jeff Hare) Date: Tue, 23 Mar 2004 20:24:10 -0500 Subject: [geeklog-devel] Login / Compact Privacy policy support Message-ID: <6.0.3.0.2.20040323201710.0349c1d8@mail.thehares.com> Hello, Is there anything that geeklog development team can (or should) do to support the new compact privacy policy protocol? It appears that it might make it easier for visitors to sign up/log in when their browser is locked down a little too tight for geeklog to write the necessary cookie info. By far, the number one problem I've experienced running portals is dealing with the login related issues of non-technical members who have overly restricted their browser... Just a thought. I'll continue to look into this. -Jeff Hare From vmf at abtech.org Sat Mar 27 14:54:13 2004 From: vmf at abtech.org (Vincent Furia) Date: Sat, 27 Mar 2004 14:54:13 -0500 Subject: [geeklog-devel] Comment Speed Improvement Message-ID: <4065DBE5.5070908@abtech.org> I've been thinking hard about improving the speed of displaying comments by reducing the number of queries and potentially removing the need to use recursive functions. While Niels solution provides a quick fix for groklaw's specific problems, it is pretty inefficient when you are looking at a subset of comments for a story. A more elegant solution must be possilbe.. I think I've found that solution, but it will take some work (including a database change) to implement. While I'm fully capable of implementing it, I wanted to get some feed-back before jumping in. The idea is to store the comments in the database using a modified preorder tree traversal method. A brief description (along with a comparison to our current method) can be found in this article: http://www.sitepoint.com/article/hierarchical-data-database A better description is this site: http://membres.lycos.fr/sqlpro/Tree/SQL_tree.htm However it is in French (which I don't speak). But babelfish (http://babel.altavista.com) works pretty well on the first half of the page. Plus it has many useful graphics. The idea is a little confusing at first, but once you wrap your head around it you'll think: "Wow, hey, that's pretty darn cool". Or maybe you'll think: "Mein Gott, das ist wirklich k?hl.". ;) The summary: - 1 query and 2 updates to insert a new comment - 1 query to get all the comments (in a logical order for display) - 1 query to get all the children of a given comment (with or without the given comment) - 1 query to count the number of children a comment has Basically this will drastically improve the speed of comment displays of all types, especially for sites that have very large databases of comments (groklaw). The only expense is adding two extra db queries when a comment gets added (including one that could potentially update almost every database row). The only change to the database will be the addition of two integer columns: left terminal (lft) and right terminal (rgt). Optionally we could also remove the parent id (pid) column as it would no longer be technically necessary. That being said, I think we should preserve it for convenience. I think the improvement will be worth it, though we'll have to write a slick upgrade script to take care of current comments. A side effect of using this method is that we will be able to write a comment function that is iterative rather than recursive which will further improve performance. Another note is that this method is database independent. There are some other pretty nice solutions that require stored procedures and I think Oracle has something built in. Obviously those other solutions are really going to work well with MySQL. If the article above doesn't make sense, let me know and I can write a brief summary of what is going on in "preorder tree traversal" method. Please everyone, let me know what you think. I'd especially like some input from a DB guy to make sure I'm not missing something obvious (Dwight, you still out there?). -Vinny From dwight at trumbower.com Sun Mar 28 01:02:35 2004 From: dwight at trumbower.com (Dwight Trumbower) Date: Sun, 28 Mar 2004 00:02:35 -0600 Subject: [geeklog-devel] Comment Speed Improvement In-Reply-To: <4065DBE5.5070908@abtech.org> References: <4065DBE5.5070908@abtech.org> Message-ID: <6.0.3.0.0.20040328000051.01c04ed0@localhost> English version of the same method can be found here. http://www.intelligententerprise.com/001020/celko.jhtml?_requestid=13769 It is by Joe Celko, one of the gurus in SQL programming. He has been promoting this method for years. I have used this method before and it works nice. At 01:54 PM 3/27/2004, you wrote: >I've been thinking hard about improving the speed of displaying comments >by reducing the number of queries and potentially removing the need to use >recursive functions. > >While Niels solution provides a quick fix for groklaw's specific problems, >it is pretty inefficient when you are looking at a subset of comments for >a story. A more elegant solution must be possilbe.. > >I think I've found that solution, but it will take some work (including a >database change) to implement. While I'm fully capable of implementing >it, I wanted to get some feed-back before jumping in. > >The idea is to store the comments in the database using a modified >preorder tree traversal method. A brief description (along with a >comparison to our current method) can be found in this article: > > http://www.sitepoint.com/article/hierarchical-data-database > >A better description is this site: > > http://membres.lycos.fr/sqlpro/Tree/SQL_tree.htm > >However it is in French (which I don't speak). But babelfish >(http://babel.altavista.com) works pretty well on the first half of the >page. Plus it has many useful graphics. > >The idea is a little confusing at first, but once you wrap your head >around it you'll think: "Wow, hey, that's pretty darn cool". Or maybe >you'll think: "Mein Gott, das ist wirklich k?hl.". ;) > >The summary: > >- 1 query and 2 updates to insert a new comment >- 1 query to get all the comments (in a logical order for display) >- 1 query to get all the children of a given comment (with or without the >given comment) >- 1 query to count the number of children a comment has > >Basically this will drastically improve the speed of comment displays of >all types, especially for sites that have very large databases of comments >(groklaw). The only expense is adding two extra db queries when a comment >gets added (including one that could potentially update almost every >database row). > >The only change to the database will be the addition of two integer >columns: left terminal (lft) and right terminal (rgt). Optionally we >could also remove the parent id (pid) column as it would no longer be >technically necessary. That being said, I think we should preserve it for >convenience. > >I think the improvement will be worth it, though we'll have to write a >slick upgrade script to take care of current comments. A side effect of >using this method is that we will be able to write a comment function that >is iterative rather than recursive which will further improve performance. > >Another note is that this method is database independent. There are some >other pretty nice solutions that require stored procedures and I think >Oracle has something built in. Obviously those other solutions are really >going to work well with MySQL. > >If the article above doesn't make sense, let me know and I can write a >brief summary of what is going on in "preorder tree traversal" method. > >Please everyone, let me know what you think. I'd especially like some >input from a DB guy to make sure I'm not missing something obvious >(Dwight, you still out there?). > >-Vinny >_______________________________________________ >geeklog-devel mailing list >geeklog-devel at lists.geeklog.net >http://lists.geeklog.net/listinfo/geeklog-devel > > > From dirk at haun-online.de Sun Mar 28 17:03:37 2004 From: dirk at haun-online.de (Dirk Haun) Date: Sun, 28 Mar 2004 23:03:37 +0100 Subject: [geeklog-devel] Comment Speed Improvement In-Reply-To: <4065DBE5.5070908@abtech.org> References: <4065DBE5.5070908@abtech.org> Message-ID: <20040328220337.22226@smtp.haun-online.de> >I've been thinking hard about improving the speed of displaying comments >by reducing the number of queries and potentially removing the need to >use recursive functions. Well, I didn't RTFA, but if it improves the speed while retaining compatibility with old links, I'm all for it. And if you could throw in links to individual comments (so that search results in comments can point to the actual comment) ... Btw, did you see that I assigned you a bug report submitted by Niels: I think for 1.3.10, we should concentrate on speed improvements (like the ones suggested by Niels, but there may be others) as well as fixing some of the inconsistencies that have crept in over the last few releases. And if we could do all that in less than 8 months this time ... Which doesn't mean that new features aren't welcome. Tony, Blaine, what's the status of your latest additions (spell checker, archiving)? bye, Dirk -- http://www.haun-online.de/ http://www.tinyweb.de/ From dirk at haun-online.de Mon Mar 29 15:20:07 2004 From: dirk at haun-online.de (Dirk Haun) Date: Mon, 29 Mar 2004 21:20:07 +0100 Subject: [geeklog-devel] Credits to ourselves? Message-ID: <20040329202007.13985@smtp.haun-online.de> With the addition of Vinny, there are now four people with CVS access contributing code on a (more or less) regular basis. Which brings up the issue of proper credits. I've always tried to properly credit code, fixes, and other things contributed by people outside of the core team. But what about the four of us now? Should we start putting our names after each change description? The PHP team does it like this, for example: Fixed getopt() so it works without $_SERVER (Rasmus, bfrance) Personally, I don't care. I can always get the information I need (who made a change a when) from CVS. But I don't want to downplay anybody's role in this project or pocket someone's credits. So, do we add our names to docs/history or do we stick with the rule that changes without credits were made by an (unnamed) team member? bye, Dirk -- http://www.haun-online.de/ http://mypod.de/ From slord at marelina.com Mon Mar 29 23:15:30 2004 From: slord at marelina.com (Simon Lord) Date: Mon, 29 Mar 2004 23:15:30 -0500 Subject: [geeklog-devel] Credits to ourselves? In-Reply-To: <20040329202007.13985@smtp.haun-online.de> References: <20040329202007.13985@smtp.haun-online.de> Message-ID: I'm not even contributing to CVS other than the glpro theme, but it seems to me that you guys need to stick your names in there. If nothing else, it's a great resume piece down the road. GL is not some hack cms, you guys have some crazy classes in there etc that may prove you're the right man for the job at your next interview. But if your names are not in CVS you'll have a hell of a time convincing them you wrote a file uploader class etc. I'm sure if I was a contributor to SAMBA I'd want my name in there. Cripes, here's a great free CMS system. The least *all* of us can do is allow the major contributors to stamp their names in there somewhere. Module developers can stick their names into their modules and theme developers can stick theirs into the css file... ;) My two cents. On Mar 29, 2004, at 3:20 PM, Dirk Haun wrote: > With the addition of Vinny, there are now four people with CVS access > contributing code on a (more or less) regular basis. Which brings up > the > issue of proper credits. > > I've always tried to properly credit code, fixes, and other things > contributed by people outside of the core team. But what about the four > of us now? Should we start putting our names after each change > description? The PHP team does it like this, for example: > > Fixed getopt() so it works without $_SERVER (Rasmus, bfrance) > > Personally, I don't care. I can always get the information I need (who > made a change a when) from CVS. But I don't want to downplay anybody's > role in this project or pocket someone's credits. > > So, do we add our names to docs/history or do we stick with the rule > that > changes without credits were made by an (unnamed) team member? > > bye, Dirk > > > -- > http://www.haun-online.de/ > http://mypod.de/ > > _______________________________________________ > geeklog-devel mailing list > geeklog-devel at lists.geeklog.net > http://lists.geeklog.net/listinfo/geeklog-devel > > Sincerely, Simon From geeklog at langfamily.ca Tue Mar 30 09:55:31 2004 From: geeklog at langfamily.ca (Blaine Lang) Date: Tue, 30 Mar 2004 09:55:31 -0500 Subject: [geeklog-devel] Comment Speed Improvement References: <4065DBE5.5070908@abtech.org> Message-ID: <001f01c41667$0cea22d0$650a10ac@XPBL2> I had not seen the "modified preorder tree traversal method" explained before but the article on Sitepoint and the one pointed by Dwight explained it pretty well. This would also work nicely for the forums as I currently use a parent/child tree model. I think it would be a good idea to try this structure and test it on a few large databases. Blaine ----- Original Message ----- From: "Vincent Furia" To: Sent: Saturday, March 27, 2004 2:54 PM Subject: [geeklog-devel] Comment Speed Improvement > I've been thinking hard about improving the speed of displaying comments > by reducing the number of queries and potentially removing the need to > use recursive functions. > > While Niels solution provides a quick fix for groklaw's specific > problems, it is pretty inefficient when you are looking at a subset of > comments for a story. A more elegant solution must be possilbe.. > > I think I've found that solution, but it will take some work (including > a database change) to implement. While I'm fully capable of > implementing it, I wanted to get some feed-back before jumping in. > > The idea is to store the comments in the database using a modified > preorder tree traversal method. A brief description (along with a > comparison to our current method) can be found in this article: > > http://www.sitepoint.com/article/hierarchical-data-database > > A better description is this site: > > http://membres.lycos.fr/sqlpro/Tree/SQL_tree.htm > > However it is in French (which I don't speak). But babelfish > (http://babel.altavista.com) works pretty well on the first half of the > page. Plus it has many useful graphics. > > The idea is a little confusing at first, but once you wrap your head > around it you'll think: "Wow, hey, that's pretty darn cool". Or maybe > you'll think: "Mein Gott, das ist wirklich k?hl.". ;) > > The summary: > > - 1 query and 2 updates to insert a new comment > - 1 query to get all the comments (in a logical order for display) > - 1 query to get all the children of a given comment (with or without > the given comment) > - 1 query to count the number of children a comment has > > Basically this will drastically improve the speed of comment displays of > all types, especially for sites that have very large databases of > comments (groklaw). The only expense is adding two extra db queries > when a comment gets added (including one that could potentially update > almost every database row). > > The only change to the database will be the addition of two integer > columns: left terminal (lft) and right terminal (rgt). Optionally we > could also remove the parent id (pid) column as it would no longer be > technically necessary. That being said, I think we should preserve it > for convenience. > > I think the improvement will be worth it, though we'll have to write a > slick upgrade script to take care of current comments. A side effect of > using this method is that we will be able to write a comment function > that is iterative rather than recursive which will further improve > performance. > > Another note is that this method is database independent. There are > some other pretty nice solutions that require stored procedures and I > think Oracle has something built in. Obviously those other solutions > are really going to work well with MySQL. > > If the article above doesn't make sense, let me know and I can write a > brief summary of what is going on in "preorder tree traversal" method. > > Please everyone, let me know what you think. I'd especially like some > input from a DB guy to make sure I'm not missing something obvious > (Dwight, you still out there?). > > -Vinny > _______________________________________________ > geeklog-devel mailing list > geeklog-devel at lists.geeklog.net > http://lists.geeklog.net/listinfo/geeklog-devel > From geeklog at langfamily.ca Tue Mar 30 10:03:01 2004 From: geeklog at langfamily.ca (Blaine Lang) Date: Tue, 30 Mar 2004 10:03:01 -0500 Subject: [geeklog-devel] Credits to ourselves? References: <20040329202007.13985@smtp.haun-online.de> Message-ID: <002901c41668$185cc7c0$650a10ac@XPBL2> Dirk, are you thinking there would be a 1 line comment added to the changed file header documenting the change and author? I agree that a comment standard with the develper name is a good idea. The question is do we want to really add this to the header of each file - I suspect no. My changes are mostly noted in the history doc and I have not had a problem with the current method. Blaine ----- Original Message ----- From: "Dirk Haun" To: Sent: Monday, March 29, 2004 3:20 PM Subject: [geeklog-devel] Credits to ourselves? > With the addition of Vinny, there are now four people with CVS access > contributing code on a (more or less) regular basis. Which brings up the > issue of proper credits. > > I've always tried to properly credit code, fixes, and other things > contributed by people outside of the core team. But what about the four > of us now? Should we start putting our names after each change > description? The PHP team does it like this, for example: > > Fixed getopt() so it works without $_SERVER (Rasmus, bfrance) > > Personally, I don't care. I can always get the information I need (who > made a change a when) from CVS. But I don't want to downplay anybody's > role in this project or pocket someone's credits. > > So, do we add our names to docs/history or do we stick with the rule that > changes without credits were made by an (unnamed) team member? > > bye, Dirk > > > -- > http://www.haun-online.de/ > http://mypod.de/ > > _______________________________________________ > geeklog-devel mailing list > geeklog-devel at lists.geeklog.net > http://lists.geeklog.net/listinfo/geeklog-devel From dirk at haun-online.de Tue Mar 30 12:16:31 2004 From: dirk at haun-online.de (Dirk Haun) Date: Tue, 30 Mar 2004 19:16:31 +0200 Subject: [geeklog-devel] Credits to ourselves? In-Reply-To: <002901c41668$185cc7c0$650a10ac@XPBL2> References: <002901c41668$185cc7c0$650a10ac@XPBL2> Message-ID: <20040330171631.12126@smtp.haun-online.de> Blaine, >Dirk, are you thinking there would be a 1 line comment added to the changed >file header documenting the change and author? No, of course not [1]. I was talking about the history file, where we currently have entries like - Make sure the user's preferred comment limit is used when changing the comment display via the comment bar (bug #176). - The static pages editor doesn't display a "delete" button for new and cloned pages any more (since, obviously, you can't delete what hasn't been saved yet ...). from which you can't tell who did them. bye, Dirk [1] At work, there is this annoying habit of using the CVS $Log$ tag in the header of each source file, which means that you have to scroll through - literally - hundreds of lines of old log entries (most of which read "empty log message" anyway ...) before you can see the actual source code. I've silently started to use $Id$ for new files ... -- http://www.haun-online.de/ http://geeklog.info/ From vmf at abtech.org Tue Mar 30 12:33:06 2004 From: vmf at abtech.org (Vincent Furia) Date: Tue, 30 Mar 2004 12:33:06 -0500 Subject: [geeklog-devel] Credits to ourselves? In-Reply-To: <20040329202007.13985@smtp.haun-online.de> References: <20040329202007.13985@smtp.haun-online.de> Message-ID: <4069AF52.2010202@abtech.org> Sounds fine to me. That way everyone can see what slackers the rest of us are compared to Dirk. ;) A quick related question for Dirk: I noticed that most of the changes to the "history" file are by you, do you want contributers to update the file as they fix things, or do you prefer to add to it yourself? -Vinny Dirk Haun wrote: >With the addition of Vinny, there are now four people with CVS access >contributing code on a (more or less) regular basis. Which brings up the >issue of proper credits. > >I've always tried to properly credit code, fixes, and other things >contributed by people outside of the core team. But what about the four >of us now? Should we start putting our names after each change >description? The PHP team does it like this, for example: > > Fixed getopt() so it works without $_SERVER (Rasmus, bfrance) > >Personally, I don't care. I can always get the information I need (who >made a change a when) from CVS. But I don't want to downplay anybody's >role in this project or pocket someone's credits. > >So, do we add our names to docs/history or do we stick with the rule that >changes without credits were made by an (unnamed) team member? > >bye, Dirk > > > > From dirk at haun-online.de Tue Mar 30 12:59:07 2004 From: dirk at haun-online.de (Dirk Haun) Date: Tue, 30 Mar 2004 19:59:07 +0200 Subject: [geeklog-devel] Credits to ourselves? In-Reply-To: <4069AF52.2010202@abtech.org> References: <4069AF52.2010202@abtech.org> Message-ID: <20040330175907.15237@smtp.haun-online.de> Vinny, >A quick related question for Dirk: I noticed that most of the changes >to the "history" file are by you, do you want contributers to update the >file as they fix things, or do you prefer to add to it yourself? It would be nice if you would add them yourself. With the CVS notifications still not working, I may miss some changes otherwise (and, of course, it's less work for me). I always try to keep the history file at 80 characters per line, with the newest changes at the top (although I do rearrange things occassionally). bye, Dirk -- http://www.haun-online.de/ http://www.macosx-faq.de/ From tony at tonybibbs.com Wed Mar 31 00:56:53 2004 From: tony at tonybibbs.com (Tony Bibbs) Date: Tue, 30 Mar 2004 23:56:53 -0600 Subject: [geeklog-devel] Comment Speed Improvement In-Reply-To: <20040328220337.22226@smtp.haun-online.de> References: <4065DBE5.5070908@abtech.org> <20040328220337.22226@smtp.haun-online.de> Message-ID: <406A5DA5.8050103@tonybibbs.com> Spell checker is basically ready to go with some rigorous testing. Again, it is important to note that aspell doesn't like words with single quotes (e.g. don't, won't, can't, we'll, etc). However, despite this I think the feature is useful and, as is typical with most of our features, it can be turned off. I'll see if I can't get the code into CVS in short order. Keep in mind the spell checking will only be done for stories and comments since there is an existing preview option that makes integration easier. For future release we'll have to talk about how we might want to add it to things like links, events, etc which don't have a preview concept. --Tony Dirk Haun wrote: >>I've been thinking hard about improving the speed of displaying comments >>by reducing the number of queries and potentially removing the need to >>use recursive functions. >> >> > >Well, I didn't RTFA, but if it improves the speed while retaining >compatibility with old links, I'm all for it. And if you could throw in >links to individual comments (so that search results in comments can >point to the actual comment) ... > >Btw, did you see that I assigned you a bug report submitted by Niels: > > > >I think for 1.3.10, we should concentrate on speed improvements (like the >ones suggested by Niels, but there may be others) as well as fixing some >of the inconsistencies that have crept in over the last few releases. And >if we could do all that in less than 8 months this time ... > >Which doesn't mean that new features aren't welcome. Tony, Blaine, what's >the status of your latest additions (spell checker, archiving)? > >bye, Dirk > > > >