Welcome to the KForge Install Guide.
If you have any questions or require assistance, please don’t hesitate to ask on the KForge User mailing list.
KForge is tested with Python 2.6 and Python 2.7.
Use the KForge installer to generate a KForge service in a new virtual Python environment.
The installer will download and install the KForge system, and then create a service – with a KForge config file, HTML templates, static media, an SQLite database, and an auto-generated Apache configuration file to be included in a virtual host of the main Apache configuration.
Start by checking basic system packages are installed (Debian package names).
$ sudo aptitude install apache2 libapache2-mod-wsgi sqlite3 python-pip
Install the KForge installer.
$ sudo pip install kforgeinstall
Run the KForge installer. Please note, KForge should never be installed under DocumentRoot of the Apache installation, or any other directory exposed via Apache web server.
$ kforge-install --project-data-dir=/path/to/kforgeprojectdata /path/to/kforgesystem
If the installer fails, examine its output and try to resolve the error. Once the error has been resolved, remove the newly created folders, and run the installer again.
Finally, include the generated Apache configuration file in an Apache virtual host (see below) and restart Apache.
Visit the site. Login with username ‘admin’, password ‘pass’. Does it work?
Please note, it is strongly recommended that you update this password immediately (see User Guide).
The KForge install steps are summarised here. Details are provided below.
Start by checking basic system packages are installed (Debian package names).
$ sudo aptitude install apache2 libapache2-mod-wsgi python-pip
Install common compilers, libraries, and headers files (not always necessary).
$ sudo aptitude install build-essential python-dev
Decide which database management system will be used by KForge. You can use SQLite, or PostgreSQL, or MySQL, or (probably) any other system supported by SQLObject.
If necessary, please consult the user guide or manual for your database system.
If KForge will not have access to global Python packages (perhaps because the –no-site-package option will be used to create the virtual environment), install the dev packages for your database system, since the database bindings will need to be built against the header files.
Please note, if you wish to use PostgreSQL on Debian then you will discover that the python-psycopg2 package depends on python-egenix-mxdatetime package. In this case, installing egenix-mx-base from the Python package index can cause conflicts and occasionally lead to strange errors. Of course, it is possible and sometimes necessary to install egenix-mx-base and psycopg2 locally (e.g. into a virtual environment), but it is much easier to use the system (e.g. Debian) packages. Other similar conflicts could arise, depending on what you have installed, so keep things simple if you can.
By default, KForge will use SQLite. Install SQLite unless you wish to use another database management system (such as MySQL or PostreSQL). Please note, Python bindings for SQLite are included in the core Python distribution, so there is no need to install Python bindings for SQLite.
$ sudo aptitude install sqlite3
If KForge will not have access to global Python packages, install the PostgreSQL header files.
$ sudo aptitude install libsqlite3-dev
There is no need to create a database user and database instance for SQLite.
If you wish to use PostgreSQL, then install these packages instead.
$ sudo aptitude install mysql-server python-mysqld
If KForge will not have access to global Python packages, install the PostgreSQL header files. Make sure to use the correct version of PostgreSQL headers.
$ sudo aptitude install postgresql-server-dev-8.4
Create a database user for KForge to use and set a password for the user. If necessary, configure the database system to allow this user to access the KForge database instance via TCP/IP sockets (edit the pg_hba.conf file).
Create the database instance. You can also create the database instance after the KForge configuration file has been created by using the ‘kforge-admin db create’ command (see below), but then the KForge database user must have been given permission to create databases. If you don’t want the KForge database user to be able to create databases, create the database instance now.
If you wish to use MySQL, then install these packages instead.
$ sudo aptitude install postgresql python-psycopg2
If KForge will not have access to global Python packages, install the MySQL header files.
$ sudo aptitude install libmysqlclient-dev
Create a database user for KForge to use and set a password for the user. If necessary, configure the database system to allow this user to access the KForge database instance via TCP/IP sockets.
Create the database instance. You can also create the database instance after the KForge configuration file has been created by using the ‘kforge-admin db create’ command (see below), but then the KForge database user must have been given permission to create databases. If you don’t want the KForge database user to be able to create databases, create the database instance now.
It is recommended that KForge is installed in a virtual Python environment. It is possible, but not recommended, to install KForge in your global Python packages.
Please note, KForge should never be installed under DocumentRoot of the Apache installation or any other directory exposed via Apache web server.
Install KForge either by installing and running the KForge installer, or by creating a virtual environment and installing KForge into it by hand.
After installation, check the following KForge scripts are available.
$ which kforge-makeconfig
$ which kforge-admin
The KForge installer is an application of virtualenv. It will create a new virtual Python environment and install KForge into it. A new KForge service will be set up (optionally not with the –skip-service-setup option).
If you are upgrading, then prepare the migration data file before installing the new version.
$ /path/to/old/kforge-admin --config /path/to/old/kforge.conf migratedump DUMPPATH
Install the KForge installer.
$ sudo pip install kforgeinstall
Please read the help message for information about installer options.
$ kforge-install --help
The only required argument is a filesystem path to the new KForge installation. Please note, this folder should not be under DocumentRoot of the Apache installation or any other directory exposed via Apache web server. To prepare for installing future versions of KForge, it is recommended to include the installed version of KForge in the /path/to/kforgesystem filesystem path.
$ kforge-install /path/to/kforgesystem
If you only wish to install the software, use the –skip-service-setup option. KForge services can be set up later using kforge-makeconfig and kforge-admin setup (see below).
$ kforge-install --skip-service-setup /path/to/kforgesystem
It is recommended to locate the project service data folder outside of the virtualenv installation by using the –project-data-dir option. The project service data folder will contain e.g. Subversion repositories and Trac environments. This folder will be used by later versions of KForge, so it makes good sense to separate it from this version of the KForge system.
$ kforge-install --project-data-dir=/path/to/kforgeprojectdata /path/to/kforgesystem
If you wish to set your own service name, use the –service-name option.
$ kforge-install --service-name="My Forge" /path/to/kforgesystem
If you are upgrading, use the –dump-file option to indicate where migrated data can be found. If you are using PostgreSQL of MySQL, make sure to use a different database name from that used by the existing service.
$ kforge-install --dump-file DUMPPATH /path/to/kforgesystem
If you want to boost the performance of the access controller, then enable memoization with the –enable-memoization option.
$ kforge-install --enable-memoization /path/to/kforgesystem
The installer will attempt to install necessary database bindings. Unless the –skip-service-setup option is used, the installer will also create and initialise the database. By default, an SQLite database will be created and initialised, as if the –db-type option had been set to ‘sqlite’.
$ kforge-install --db-type=sqlite /path/to/kforgesystem
If you want to use PostgreSQL instead of SQLite, use the –db-type option, along with the –db-name, –db-user and –db-pass options.
$ kforge-install --db-type=postgres --db-name=DBNAME \
--db-user=DBUSER --db-pass=DBPASS /path/to/kforgesystem
Alternatively, for MySQL the –db-type option would take value of ‘mysql’.
$ kforge-install --db-type=mysql --db-name=DBNAME \
--db-user=DBUSER --db-pass=DBPASS /path/to/kforgesystem
If the database user does not have the privilege to create the database, then create the database before running the installer and use the –skip-db-create option.
$ kforge-install --skip-db-create --db-type=postgres \
--db-name=DBNAME --db-user=DBUSER --db-pass=DBPASS /path/to/kforgesystem
If you wish to install KForge into an isolated virtual Python environment, then use the –no-site-packages option. Please note, this option is not required to obtain a working installation, and can make installing required Python packages more difficult.
$ kforge-install --no-site-packages /path/to/kforgesystem
Whichever options you use, after running the installer try activating the new virtual Python environment.
$ source /path/to/kforgesystem/bin/activate
If you are upgrading, compare the new KForge configuration file with the existing service. Carry over any preferences for log levels, memoization, secret key, email address, and so on.
$ vim /path/to/kforgesystem/etc/kforge.conf
Create and activate a new virtual Python environment.
$ virtualenv /path/to/kforgesystem
$ source /path/to/kforgesystem/bin/activate
Install the KForge Python package with pip.
$ pip install kforge
Alternatively, download KForge, extract the distribution, and the run the setup.py file.
$ python setup.py install
If KForge was installed with the KForge installer, then a service will already have been setup (unless the –skip-service-setup option was used), so continue to the next section.
Otherwise, the environment variable KFORGE_SETTINGS can be used to indicate the filesystem path to the KForge configuration file.
$ export KFORGE_SETTINGS=/path/to/kforgesystem/etc/kforge.conf
There are lots of options for setting configuration values.
$ kforge-makeconfig --help
Create a new configuration file with the kforge-makeconfig program.
$ kforge-makeconfig --master-dir=/path/to/kforgesystem > $KFORGE_SETTINGS
Edit the configuration file with your favorite editor.
Make sure the kforge-admin program is available.
$ which kforge-admin
Use the setup command of the kforge-admin program to setup the service.
$ kforge-admin setup
Use the apacheconfig path command of the kforge-admin program to locate the auto-generated KForge Apache configuration file.
$ kforge-admin apacheconfig path
Include the auto-generated KForge Apache configuration file in your main Apache configuration, for example within the VirtualHost for the domain on which you will run KForge. As it will be regenerated as project services change, it is important to include the generated file, rather than copy its contents elsewhere.
<VirtualHost *:80>
ServerName kforge.mydomain.com
Include /path/to/kforgesystem/var/httpd-autogenerated.conf
<IfModule mod_wsgi.c>
WSGIDaemonProcess kforge threads=15 maximum-requests=1000
</IfModule>
</VirtualHost>
Give the Apache user (the user under which the web server runs) ownership of the KForge folders. There is more information on setting file permissions in the Additional Notes section below.
$ sudo chown -R www-data:www-data /path/to/kforgeprojectdata /path/to/kforgesystem
KForge modifies the Apache configuration file when project services are created, updated, and deleted. Set KForge to automatically reload Apache by setting the ‘reload_apache’ configuration variable (and commenting out the no_reload_apache option). Note that for this to work you will need to ensure appropriate permissions are granted to the user under which the KForge system runs (more details below).
Alternatively, it is possible to reload Apache using some external means. For example, create a cron job that automatically reloads Apache at regular intervals (example below) or a monitoring service of some kind (such as monit) to reload apache each time the apache config file is rebuilt (example below).
KForge is tested with Apache 2.2.
There is further information in the Additional Notes section below.
Restart Apache with the normal command line command.
$ sudo /etc/init.d/apache2ctl restart
Optionally, tail the Apache and KForge logs for errors.
$ tail -f /var/log/apache2/*log /path/to/kforgesystem/var/log/kforge.log
Attempt to visit the KForge service in your browser. Login with username ‘admin’, password ‘pass’. Does it work?
Please note, it is strongly recommended that you update this password immediately (see User Guide).
Please note, some files need to be writable by the Apache user. By default, these files are all contained within the /path/to/kforgesystem/var folder:
The remaining files need only to be readable by the Apache user:
For web access to work for KForge you must allow Apache to access the KForge files. Three options are described below.
The first option is to have the Apache user own all KForge files. This may cause problems if you then wish to run command line utilities to administer KForge, since you may not have permission to write to files and directories. You can also run the commands as the Apache user.
The second option is to run Apache as the ‘kforge’ user. This may cause problems if you wish to use the Apache server for other things, but it makes the SSH plugin easy to configure.
The third option allows for dual access by the KForge user and the Apache user:
Set umask for ‘kforge’ user so that the group has write permission. A suitable umask value UMASK to allow group access is 002 or 007. Add the following to the your environment config (e.g. ~/.bashrc):
umask UMASK
Giving Apache user access to KForge service files by adding Apache to KForge group. For example:
$ usermod -GKFORGEGROUP APACHEUSER
Set umask for Apache to allow group access.
Add the following to Apache startup script (usually /usr/sbin/apachectl or /usr/sbin/apache2ctl – you can find it referenced in /etc/init.d/apache<version>):
umask UMASK
Give group access to existing files and make sure new files and directories remain in the KForge group by setting sgid:
$ chmod g+rX -R /path/to/kforgesystem
$ chmod g+wX -R /path/to/kforgesystem/var
When project services change, it is necessary for the KForge system to regenerate the KForge Apache configuration file. For these changes to take effect it is then necessary for Apache to reload its configuration.
There are three options for reloading the Apache configuration file:
Event-driven reloading with KForge
In the [www] section of the KForge configuration file, set ‘reload_apache’ to the command which will reload the apache configuration on your system.
NB: In order for this to work the user which owns the Apache process must be able to reload the configuration. Specifically it must be able to run the ‘reload_apache’ command specified in the [www] section of the configuration. You can normally achieve this by adding something to your /etc/sudoers file:
Cmnd_Alias APACHE = <insert-apache-reload-command>
[....]
<www-user-name> ALL =NOPASSWD: APACHE
Regular time-interval reloading with external tool (e.g. cron)
Reload the Apache config on a regular basis using some external tool. For example the following uses cron to reload Apache config every half an hour:
# Reload Apache config every half an hour.
# The apache reload command is for apache2 on Debian
# and might need to be modified for other systems.
30 * * * * /etc/init.d/apache2 reload
Event-driven reloading with external tool (e.g. monit)
Alternatively you could use some form of monitoring service to only reload when KForge rebuilds its Apache configuration file. For example, the following uses monit:
check file ${file-name} with path ${path-to-kforge-generated-apache-config}
if changed timestamp
then exec "/etc/init.d/apache2 reload"
Backups. Hotcopy KForge model data and project service data. On an external machine, rsync both the installation and the hotcopy folders. You may also dump raw SQL, just use a suitable database command.
# Hotcopy KForge service data every night.
0 4 * * * /path/to/kforge-admin --config=/path/to/kforge.conf backup BACKUPPATH
Recent changes. Update the recent changes feeds.
# Update KForge feed every half an hour.
0,30 * * * * /path/to/kforge-admin --config=/path/to/kforge.conf updatefeed
If you never run this command, the ‘recent changes’ feed will not show.
There is a test runner for the KForge test suite.
$ kforge-test
Todo: More information about running the KForge test suite.
Like KForge, the Provide system is an application of Domain Model. It will create a new virtual Python environment, install KForge into it, and then set up several KForge services.
It will optionally configure and restart Apache. It will test the software before production deployment, and it will allow data migrations to be verified before cutting users over to new versions.
Read about Provide, learn about the working process it supports.
<http://appropriatesoftware.net/provide/>
Download and deploy Provide.
<http://kforge.appropriatesoftware.net/provide/trac/wiki#DeployingProvide>
Deploy KForge inside Provide.
Provide a new KForge version x.xx service with these commands:
$ provide scripts kforge x.xx
$ provide-kforge-init
$ provide-kforge-x.xx-init
$ provide-kforge-x.xx-accept
$ provide-kforge-x.xx-production
Move from x.xx to y.yy with:
$ provide scripts kforge y.yy
$ provide-kforge-y.yy-init
$ provide-kforge-y.yy-accept-from-x.xx
$ provide-kforge-y.yy-production-from-x.xx
Later, move to a future KForge version z.zz with:
$ provide scripts kforge z.zz
$ provide-kforge-z.zz-init
$ provide-kforge-z.zz-accept-from-y.yy
$ provide-kforge-z.zz-production-from-y.yy
Todo: More information about using KForge with Provide.