FLAT Installation Guide¶
Installation¶
Setting up a web application is often a fairly complex endeavour. To facilitate this process we provide an OCI/Docker container image which you can use. For this we assume some basic familiarity with docker on your part:
$ docker pull proycon/flat
You can subsequently start a container as follows (minimal example):
$ docker run -p 8080:80 -v /path/to/data:/data proycon/flat
The /path/to/data
is the data on the host system you want to mount into the container. This is where all FoLiA
documents will be stored. Make sure you have write permission here; the FLAT container will use UID and GID 100 internally (by default), which will translate to higher UIDS/GIDS on the host based on
your subordinate user/group configuration, you may need to need to remap subordinate user/group IDs .
The setting -p 8080:80
maps port 8080 on the host to 80 in the container,
so after the container is started you can navigate to
https://localhost:8080
. By default you can login with user flat
and password flat
if you haven’t made any modifications to the default configuration.
Usage of our container images is the recommended way to install in production
environments. The container accepts a wide variety of environment variables to
configure FLAT. See the Dockerfile for the various
variables you can set, you can subsequently pass them to docker run
using
--env
(can be passed multiple times), for example:
$ docker run -p 8080:80 -v /path/to/data:/data --env FLAT_REVERSE_PROXY_HTTPS=1 proycon/flat
In order to configure FLAT for specific annotation tasks we recommend you use
of external YAML files. The format of these is described in the FLAT
Administration Guide.
Once you have a directory on your host (e.g. /path/to/config) containing one or more such
external YAML files (use extension .yml, an example is provided here), you can make them available inside the container by
setting FLAT_CONFIG_DIR
and mounting another volume as follows:
$ docker run -p 8080:80 -v /path/to/data:/data -v /path/to/config:/etc/flat --env FLAT_CONFIG_DIR=/etc/flat proycon/flat
Manual Installatiion¶
FLAT runs on Python 3. If you don’t use the container image, we recommend installation in a Python virtualenv with Python 3, create one as follows:
$ virtualenv --python=python3 env
Activate the virtual environment as follows (you will need to do this every time you want to access the virtual environment):
$ . env/bin/activate
FLAT can then simply be installed through the Python Package Index:
$ pip install FoLiA-Linguistic-Annotation-Tool
Or alternatively from the cloned git repository:
$ pip install -r requires.txt #(to install the dependencies)
$ python3 setup.py install
If you are the system administrator and opt for a global installation instead
of using a virtualenv, then just add sudo
on most Linux distributions.
The following dependencies will be pulled in automatically if you follow either of the above steps:
- foliadocserve (https://github.com/proycon/foliadocserve)
- foliapy (https://github.com/proycon/foliapy)
- django (https://www.djangoproject.com)
Troubleshooting: If the pip installation procedure fails because of libxml2 issues, you are missing one or more global dependencies. Make to install the following through your distribution’s package manager:
$ apt-get install libxml2 libxml2-dev libxslt1.1-dev libxslt1
Then run pip install FoLiA-Linguistic-Annotation-Tool
again. Note that the
above instruction is geared towards Debian/Ubuntu systems, look up the
corresponding package names and invoke your distribution’s package manager if
you are on another system.
Upgrade¶
If you use the container images as recommended, just pull in the latest image version from Docker Hub and you can ignore the rest of this section.
To upgrade your existing manual installation of flat to the latest version, run the following from within your virtual environment:
$ pip install -U FoLiA-Linguistic-Annotation-Tool
New versions of FLAT may introduce new configuration options for your
settings.py
(introduced in next section). Please inspect the differences
between your variant of settings.py
and the one provided with FLAT, and
copy what is needed. If you use external YAML configuration files, you likely won’t have this problem.
New versions may also introduce database migrations. To
run these, set PYTHONPATH
to the directory that contains your
settings.py
, and DJANGO_SETTINGS_MODULE
to the name of the file without the extension:
$ export PYTHONPATH=/your/settings/path/
$ export DJANGO_SETTINGS_MODULE=settings
$ django-admin migrate --run-syncdb
Don’t forget to restart your webserver or development server, as well as the
FoLiA Document Server (foliadocserve
) after each upgrade.
FLAT Configuration¶
You can configure FLAT either by editing settings.py
, or by passing environment variables and some external YAML
files at run time, the latter is recommended as it means you can use the container image as-is.
If you do use settings.py
directly, download a copy from
https://raw.githubusercontent.com/proycon/flat/master/settings.py to some
custom location, edit it and add a configuration for your system. The file is
heavily commented to guide you along with the configuration. It is here where
you specify what your users will see and what function are enabled.
Further documentation on configuration FLAT, after completing the installation and configuration described in the remainder of this section, can be found in the FLAT Administration Guide.
Database Configuration¶
FLAT uses a database only to store user accounts. In your settings.py
you refer
to this database. Multiple backends are supported (MySQL, PostgreSQL and
others). Make sure you create the desired database and user, with proper rights
to access and modify the database, in your database management system.
If you use the container image (as recommended), you can set environment variables to configure the database (see https://github.com/proycon/flat/blob/master/Dockerfile). In most cases you won’t need to edit anything and the default sqlite database the containers offers should be sufficient as FLAT only makes light usage of the database.
$FLAT_DATABASE_ENGINE
: can be set todjango.db.backends.sqlite3
,django.db.backends.mysql
,django.db.backends.postgresql_psycopg2
and others.$FLAT_DATABASE
: name of the database, filename in case of sqlite3$FLAT_DATABASE_USER
$FLAT_DATABASE_PASSWORD
$FLAT_DATABASE_HOST
$FLAT_DATABASE_PORT
In manual installations, before you start FLAT for the first time, the database needs to be
populated. Set PYTHONPATH
to the directory that contains your
settings.py
, and DJANGO_SETTINGS_MODULE
to the name of the file without the extension:
$ export PYTHONPATH=/your/settings/path/
$ export DJANGO_SETTINGS_MODULE=settings
$ django-admin migrate --run-syncdb
$ django-admin createsuperuser
OpenID Connect Authentication¶
Flat supports OpenID Connect as a means of authenticating with a single-sign on authentication provider. Set OIDC =
True
in settings.py
or set environment variable FLAT_OIDC=1
and configure all the other variables needed for
OpenID Connect Authentication, either by editing settings.py
or by passing the proper environment variables when
starting the container:
$FLAT_CLIENT_ID
- Client ID as registered with the OpenID Connect Provider$FLAT_CLIENT_SECRET
- Client secret as registered with the OpenID Connect Provider$FLAT_AUTH_ENDPOINT
- URL of the authorization endpoint at the OpenID Connect Provider$FLAT_TOKEN_ENDPOINT
- URL of the token endpoint at the OpenID Connect Provider$FLAT_USER_ENDPOINT
- URL of the userinfo endpoint at the OpenID Connect Provider$FLAT_TOKEN_USE_BASIC_AUTH
- Set to 1 to use client_secret_basic rather than client_secret_post (depends on your provider)$FLAT_SIGN_ALGO
- Sign algorithm your OpenID Connect provider uses, defaults to RS256 (can be set to HS256)$FLAT_JWKS_ENDPOINT
- URL of the OIDC OP JWKS endpoint, to obtain the signing key automatically$FLAT_IDP_SIGN_KEY
- The full signing key manually (alternative to the above), the content of this variable is interpreted as JSON. Example:key: { "kty": "rsa", "use": "sig", "alg": "rs256", "n": "SOME VALUE!", "e": "aqab" }
Users that are authenticated in this way are still added to the internal user database, which is needed when you want to configure groups and rights. Matching OpenID Connect users with users in the database is always done on the basis of the user’s e-mail address. FLAT will consistently use e-mail addresses to represent users if you enabled Open ID Connect.
Starting the Document Server¶
FLAT constantly talks to a document server running in the background.
We need to start the FoLiA document server prior to starting FLAT, it is a required component that needs not necessarily be on the same host. The container image we provide already contains both FLAT and the document server, so you don’t need to do anything for it and can skip this section entirely.
Your copy of settings.py
should point to the host and port where FLAT can reach the
document server, this can also be done using environment variables FOLIADOCSERVE_HOST
, FOLIADOCSERVE_PORT
and
FLAT_DOCROOT
. In manual installations you can then start it as follows:
$ foliadocserve -d /path/to/document/root -p 8080 --git
The document path will be a directory that will contain all FoLiA documents.
Create a root directory and ensure the user the foliadocserve is running under has
sufficient write permission there. The document server needs no further
configuration. Note that it does not provide any authentication features so it
should run somewhere where the outside world can NOT reach it, only FLAT needs
to be able to connect to it. Often, FLAT and the document server run on the
same host (like in our container setup), so a localhost connection is sufficient. You can set the document root using
environment variable FLAT_DOCROOT
.
The --git
option to foliadocserve
enables git versioning support, allowing users to undo annotations
and go back to previous revisions, it requires git
to be installed on the
system and your git identity to be configured:
$ git config --global user.email "you@example.com"
$ git config --global user.name "Your Name"
Starting FLAT as development server¶
If you followed the manual installation route, you can start a development server using your settings.py
by setting
PYTHONPATH
to the directory that contains it, and DJANGO_SETTINGS_MODULE
to the name of the file without the
extension:
$ export PYTHONPATH=/your/settings/path/
$ export DJANGO_SETTINGS_MODULE=settings
$ django-admin runserver
FLAT will advertise the host and port it is running on (as configured in your
settings.py
or $FOLIADOCSERVE_HOST
and $FOLIADOCSERVE_PORT
), and you can access it in your browser.
Tests¶
FLAT has integration and automatic interface tests for the annotation editor, point your
browser to http://127.0.0.1:8000/editor/testflat/testflat
to execute all tests.
Deployment in Production¶
For production environments we strongly recommend use of our container image as-is and using environment variables (and YAML files) for the configuration. This allows for nice integration in infrastructure using container orchestration platforms like kubernetes, or simpler solutions like docker compose.
A significant part of the deployment-specific configuration (database settings, authentication etc) can be configured by setting environment variables when starting the container. Here are some of the key environment variables you absolutely need to change:
$FLAT_DOMAIN
- The domain FLAT is served from (without scheme), e.g.flat.yourdomain.org
$FLAT_BASE_PREFIX
- If don’t serve under the root of the domain, set the prefix here (with leading slash ut without trailing slash)$FLAT_SECRET_KEY
- You need to set this to some random string$FLAT_USER
- The default administrative user (default: flat)$FLAT_PASSWORD
- The password for the administrative user (default: flat)$FLAT_REVERSE_PROXY_HTTPS
- You should set this to 1 after making sure you are behind a reverse proxy that handles SSL.$FLAT_CONFIG_DIR
- Set this to a directory on the/data/
mount that holds the external configuration files in YAML format. Example:/data/flat.d/
. You may omit this if you instead decided to editsettings.py
directly.$FLAT_ADMIN_NAME
- The full name of the FLAT administrator$FLAT_EMAIL
- The email address for the administrator$FLAT_OIDC
- Set this to 1 if you want OpenID Connect Authentication (see the relevant section in this documentation for the rest of this configuration).$FLAT_DEFAULTCONFIGURATION
- Set to the name of your default configuration (the filename part without the.yml
extension).$FLAT_DOCROOT
- Defaults to/data/flat.docroot
if unset, this is the directory that holds the documents and should always reside on the/data/
mount.
The remaining configurations for actual annotation tasks can be specified in the external YAML files.
If you prefer editing settings.py
rather than passing environment variables, you could build a
customised container image with your settings.py
configuration that is derived from our image. For your own
settings.py
, you can create the following simple Dockerfile
alongside it:
FROM proycon:flat
COPY settings.py /tmp/flat_settings.py
RUN cp -f /tmp/flat_settings.py /usr/lib/python3.*/site-packages/
Build your image with docker build .
and you have a docker image derived on the image we provide, which merely overrides it with your configuration.
As said before, SSL should be handled by your own reverse proxy, it’s not handled by the container. Your reverse proxy should simply handle SSL and forward all traffic to the container. The following is a reverse proxy configuration example for nginx, assuming the container is mapped to localhost on port 8080 and you have certificates ready:
server {
listen 443;
server_name flat.yourdomain.org;
ssl on;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4";
ssl_prefer_server_ciphers on;
ssl_certificate /etc/letsencrypt/live/flat.yourdomain.org/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/flat.yourdomain.org/privkey.pem;
client_max_body_size 750m;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto "https";
proxy_set_header X-Forwarded-Host $host;
proxy_pass http://127.0.0.1:8080/;
}
}
If you don’t want to use our container image or don’t want to use a reverse proxy, then you’ll have to dive a bit deeper
to get things working. In such cases we recommend using uwsgi
for serving FLAT. Apache2 users can use it with
mod_uwsgi_proxy
. Even if you don’t use our container image, studying our Dockerfile and configurations provides a
good reference for how you can set up things.