Here are the details of upgrading to a newer version of GForge. This includes a list of all our customizations to the standard GForge installation:
First get the version of GForge that we want to upgrade to:
If upgrading from CVS: -cvs export -d tmp/ -D tomorrow gforge -mv tmp gforge -tar -zcf copyover.tar.gz gforge/ -scp copyover.tar.gz gforge@cougaar.org:tmp/ else just get tarball with wget
Now we'll make a backups of the old files and move the new ones in. Recall that a PHP app is mostly just a collection of files that get parsed by the PHP interpreter:
Stop web server - /usr/local/apache/bin/apachectl stop mv /var/www/gforge-3.0/www /tmp/gforge_bkp/ Run new db/ scripts Move old www/ and common/ to upgrades dir Copy new www/ and common/ in chown -R gforge:gforge www/ common/ Copy www/.htaccess back in Copy usage/ and usage_all/ back in cp -R tmp/gforge_bkp/www/icons/ www/ cp -R tmp/gforge_bkp/www/errata/ www/
This is a CougaarForge-specific file:
mkdir www/documents && cp /tmp/gforge_bkp/www/documents/license.html www/documents
Copy in the CougaarForge custom theme:
cp -R tmp/gforge_bkp/www/themes/cougaarforge www/themes/ cp -R tmp/gforge_bkp/www/themes/savannah www/themes/ cp tmp/gforge_bkp/www/index_std.php www/
If there are any new local.inc properties, copy them in.
Restart web server so that we can test the next series of commands - you'll be adding in all the CougaarForge customizations and you'll want to check them as you go to make sure they work:
/usr/local/apache/bin/apachectl start
Now start applying customizations:
Copy www/include/features_boxes stuff back in
cp tmp/gforge_bkp/www/register/projectinfo.php www/register/
cp tmp/gforge_bkp/www/robots.txt www/
cp -R tmp/gforge_bkp/www/docs/site/ www/docs/site
www/export/index.php
- remove various broken items
www/project/admin/index.php
- remove tarballs link
- remove 'htdocs'
www/include/project_home.php
- Add Webalizer link
- Add array and blurb for link to Bugzilla
www/download.php
- Header("Content-type: application/octet-stream");
Base.tab
- "New user with SSL" -> New user
www/include/features_boxes.php
- Remove "Highest Ranked Users"
- Add Webalizer link to usage_all
- Remove group_id = 7 from top downloads
www/scm/index.php
- delete the beta viewers
- hardcoded user_id == 102, blah. (?)
www/scm/others-files-in-this-directory
- chmod to 600 or remove
common/search/SearchQuery.class
- change seach length from 3 to 2
- change line 96 to removed % from filtered characters hash
License clickthru
- modify www/frs/showfiles.php so that each file link goes to clickthru.php
- note exclusion of group_id=7
- copy www/frs/clickthru.php /var/gforge-3.0/www/frs/
- note that license URL and enable/disable are specified in /etc/gforge/local.inc
www/themes/savannah/menu.php
- copy in code from usemodwiki plugin groupmenu hook
www/docman/view.php
- tweak types to ensure PDF mime type is set when appropriate
================================
To find changed stuff:
find /var/www/gforge-3.0/ -type f -ctime +30
Note that we also have a few items in /etc/rc.d/rc.local:
# Ensure PostgreSQL restarts if the machine crashed # and left the old pid rm /var/pgsql/data/postmaster.pid su -l postgres /var/pgsql/bin/mypg_start # Start Mailman, but only after MailScanner has started /usr/local/mailman/bin/mailmanctl start
GForge requires a bunch of cronjobs to run correctly. They are currently in the root user's cron tab:
| Purpose | Crontab entry |
|---|---|
| # Run Webalizer stats | 50 23 * * * ~/webalizer/generate_all.sh |
| # MySQL database nightly dumps | 00 01 * * * cd /root/mysql/ ; ./mysql_bkp.pl |
| # PostgreSQL database nightly dumps | 10 01 * * * cd /root/pgsql/ ; ./pgsql_bkp.rb |
| # Bugzilla whiner | 00 00 * * 6 cd /var/www/bugzilla/ ; ./whineatnews.pl |
| # Update errata web page | 0 0 * * * ~/update_errata.rb |
| # Check for pending mass mailings | 0,10,20,30,40,50 * * * * ~/gforge/massmail.php |
| # Sync Unix accts with GForge accts | 00 * * * * ~/gforge/update_unix_accounts.php > /dev/null |
| # Check for new mailing lists | 00 * * * * ~/gforge/mailing_list_check.php |
| # recalculate user popularity metric | 10 1 * * * ~/gforge/calculate_user_metric.php |
| # Detailed CVS stats | 10 1 * * * ~/gforge/history_parse.php |
| # daily project_weekly_metric recalc | 12 1 * * * ~/gforge/project_weekly_metric.php |
| # daily deletion of sessions, closing jobs, etc | 15 1 * * * ~/gforge/project_cleanup.php |
| # daily crunching of survey data and other associated ratings | 41 1 * * * ~/gforge/rating_stats.php |
| # daily crunching of project summary data (counts) | 45 1 * * * ~/gforge/db_project_sums.php |
| # daily recalculate of the sums under the trove map | 50 1 * * * ~/gforge/db_trove_maint.php |
| # daily close pending artifacts | 57 1 * * * ~/gforge/check_stale_tracker_items.php |
| # Stats aggregation work | 15 07 * * * ~/gforge/db_stats_agg.php |
| # daily rotation of the activity_log | 0 0 * * * ~/gforge/rotate_activity.php |
| # daily aggregating of the numbers | 15 0 * * * ~/gforge/site_stats.php |
| # Internal database maintenance (defrag, query plans, etc) | 0 2 * * * ~/gforge/vacuum.php |
Here are the current version of various key software packages running on CougaarForge:
GForge:
Bugzilla:
When a user submits a new project, it goes into a queue. There's a procedural process to approve the project and then a technical process to actually build all the system resources associated with the project.
Procedurally, all of the site admins (currently Aaron Helsinger, Rich Lazarus, Mark Barger, and Tom Copeland) get an email when a new project proposal is submitted. One of them then forwards it to the support-csc-info mailing list for discussion.
If the proposal is rejected, one of the site admins then goes to the "Pending Projects" admin page and clicks the reject button. If it's accepted, some who has root privileges needs to:
Hi Rich - OK, the CougaarDOOM project should be all set: http://cougaar.org/projects/cougaardoom/ You might want to read the "new project admin" checklist: http://cougaar.org/docman/?group_id=7 If you have any questions, please post to the support forum: http://cougaar.org/forum/forum.php?forum_id=6 Thanks! Yours, Aaron Helsinger CougaarForge admin team
Note that the project.rb script does all the heavy lifting here. It will create the unix group, the user account (if necessary), the virtual host, the CVS repository, the project web space, the entry in viewcvs.conf, the per-project Wiki, and the Webalizer configuration file.
When a user enters a new mailing list into the GForge user interface, the new list is not immediately created. Instead, the "mailing_list_check.php" script noted above runs every 10 minutes and, if it finds a new list creation request, emails the root user. When the root user gets the email, he should:
[root@cougaar tom]# ~/gforge/mailing_list_check.php
/usr/local/mailman/bin/newlist cougaardoom-users fred@bbn.com
[root@cougaar tom]# newaliases /etc/aliases: 467 aliases, longest 71 bytes, 32346 bytes total [root@cougaar tom]#This reads in the new list aliases and sends a SIGHUP to Sendmail.
When a person is added to a project, he's not immediately placed in the corresponding Unix group. Instead, the appropriate records are inserted into the GForge tables, and the change gets reflected in the Unix groups when the hourly cronjob "update_unix_accounts.php" runs. If you want to give the person immediate checkin access, SSH in and run this cronjob manually:
[tom@cougaar ~]$ su Password: [root@cougaar tom]# cd ~/gforge/ [root@cougaar gforge]# ./update_unix_accounts.php [root@cougaar gforge]#
Here's a rough overview of the steps I took to upgrade CougaarForge to PostgreSQL 7.4.5 (from 7.3.2) and PHP 4.3.8 (from 4.2.3) and also move from RPMs to locally compiled versions: Compiling PostgreSQL - Required newer version of Bison; downloaded bison-1.875.tar.gz, ./configure && make && make install - Untarr'd postgresql-7.4.5.tar.gz into /usr/local/src - ./configure --prefix=/var/pgsql && make && make install Compiling PHP - Untarr'd php-4.3.8.tar.gz into /usr/local/src - ./configure --enable-cli --with-apache=../apache_1.3.31 --with-pgsql=/usr/local/pgsql --with-gd --with-zlib-dir=/usr/include -- oops, needs libpng-devel, installed libpng-devel-1.2.2-8.i386.rpm, that fixed it -- make && make install Recompiling Apache - recompiled Apache to make sure new PHP module was all set -- also moving Apache layout from GNU to Apache -- ./configure --with-layout=Apache --enable-shared=max --activate-module=src/modules/php4/libphp4.a --enable-shared=php4 -- make && make install Shutdown webserver Backup database - took 2 minutes, 35 seconds, created a 411 MB tar file - /etc/rc.d/init.d/postgresql stop Uninstall old PG version - rpm -e postgresql postgresql-devel postgresql-server postgresql-docs postgresql-libs php-pgsql Start up new DB engine - edit /etc/profile -- add "PGDATA=/var/pgsql/data" and export -- add /var/pgsql/bin/ to PATH /usr/sbin/groupadd postgres /usr/sbin/useradd -g postgres postgres mkdir /var/pgsql/data && chown postgres:postgres /var/pgsql/data && chmod g+s /var/pgsql/data touch /var/log/postgresql.log && chown postgres:postgres /var/log/postgresql.log su -l postgres /var/pgsql/bin/initdb -D /var/pgsql/data edit /var/pgsql/data/pg_hba.conf - remove IPv6 record su -l postgres run /var/pgsql/bin/mypg_start createuser gforge -W createdb -U gforge gforge pg_restore -d gforge < /var/backups/pgsql/Fri.tar.gz - took 5 minutes, 46 seconds Started up webserver - edited httpd.conf, moved logs to /var/log/httpd/ - removed front page news item - deleted webserver files from old gnu-style locations - updated /etc/webalizer/* to point to new logs
3/23/2005: Consolidated several documents; added a stylesheet reference; fixed some anchors. [TLC]
3/22/2005: Initial draft; mostly linking out to other documents. We'll probably consolidate those into this one. [TLC]