Table of Contents
1. Overview
The following tutorial is a complete, detailed and tested set of instructions to install an XMPP-based messaging service (
) [1, 2] using the PROSODY [1] server as a backend and the MOVIM [4, 5] client as web browser-based frontend. This installation corresponds to company-owned social messengers, such as the XMPP-based WhatsApp service [6], however, the used software for the presented solution is open-source and federated allowing communication between different citizen-owned servers and domains (similar to e-mail).This architecture allows to separate various components of a federation of social networks based on XMPP. First, a user can create (register, sign up) an account at any XMPP-based server (eg, movim.eu or others). Second, the user can use any stand-alone or web-based client (eg, mov.im or other servers with a MOVIM client installation) to access and use his/her XMPP account. Third, to represent the federation of servers with a web-based MOVIM client installation, the website MOVIM (https://movim.eu) allows to register such servers to create a list of federated servers using the MOVIM web-based client. This creates an eco-system of a social network, which allows the user to decide where to create his XMPP account and how to access his account, thereby, avoding the limitations of centralized social networks.
This tutorial intends to lower the technical barrier for decentralized, federated social networks to democratize social communication. It was inspired by the DANDELION HUB, a action mapper for civil political action with an associated XMPP-based social network (DHUB IM), to allow activists to connect in action-specific groups (“channels”, “groups”) to discuss and plan civil political actions.
Note:
- The official spelling of the MOVIM software is “Movim“. The software name has been capitalized in text for readability.
- The offical spelling of the PROSODY software is “Prosody“. The software name has been capitalized in text for readability.
Steps:
- Rent Cloud server (with IP address)
- Rent Domain and configure DNS server (forward domain to IP address)
- Prepare Operating System
- Firewall & NGINX” web server
- Installation
- Configuration
- “PROSODY” XMPP server
- Installation
- Configuration
- “MOVIM” XMPP web client
- Installation
- Configuration
- TURN (coturn) audio & video server
- Installation
- Configuration
Note: The configuration files for NGINX, PROSODY, MOVIM and TURN (coturn) are available at: https://github.com/wiligl/xmpp-config
remote> sudo apt install git gh
remote> cd /home/xmppusr/;
remote> git clone https://github.com/wiligl/xmpp-configs.git
remote> sudo gh auth login; gh repo clone wiligl/xmpp-configs # optional/alternative with authentication: run this in terminal (not emacs terminal) and use "ssh" (not https) as connection method
Note: If you copy the provided config files instead of manually editing the original files, you may have to reset owner and group and set permissions. Create an a seaparate user, eg xmppuser, to run any services, do not run them as root/admin.
remote> chown -R <user>:<group> /path/to/config/ # example, set owner and group recursively # user and group may depend on the specific service accessing the config file, hidden folders & files, eg .git/ must set explicitly
remote> chmod -R ug+rwx /path/to/config/ # exampe, set permissions recursively, hidden folders & files, eg .git/ must set explicitly
2. Hardware
I have rented the follow cloud server from cloud service provider hetzner.com:
- Configuration of HETZNER, CPX11 server:
- Location: eu-central
- Image: Debian 12
- Type: Standard, CPX11
- 2 virtual CPUs (AMD)
- 2GB Memory
- 40GB Storage
- Networking: IPv4, IPv6
- SSH keys: none
- Volumes: none
- Firewalls: none (define later, see below)
- Backups: none
- Placement group: create “XMPP_Group”
- Labels: skip
- Cloud config: skip
- Labels: skip
- Name: debian-2gb-nbg1-1 (or any other name you want)
- Costs around ~5EUR/Month
3. Domain
I have rented the domain ddhub.im from hosting.de, which accidentally offered the desired domain name for a good price. Alternative domain name providers are namecheap.com (or again hetzner.com > DNS Console). Differences in domain availability and pricing apply. Point the domain to the IP address of your cloud server (see section 2).
Figure 1: Configuration of DNS server for main domain ddhub.im (“A” record) and subdomain turn.ddhub.im (“A” (IPV4) record and “AAAA” (IPV6, global scope) record).
4. Prepare Operating System
After renting and starting your cloud server, you will have to create a 1) root password and 2) login into the server via the VNC terminal on the provider website. Please note that the keyboard layout on the terminal may be different from your physical keyboard and therefore it it is recommended to copy and paste the root password and only use ASCII characters (no special characters or umlauts) for passwords in the VNC terminal. One can change the passwords to more complex passwords later.
Figure 2: Reset root password
Figure 3: Start the VNC terminal using the “>_” logo (see right side of screenshot) to login in as root
Update the Operating system:
remote> passwd # re-set root password to a memorable password (optional)
remote> dpkg-reconfigure locales # configure locale of language (choose english "en_US.UTF-8" for easier debugging)
remote> apt update # update package repository
remote> apt upgrade # upgrade packages
remote> apt install sudo # install sudo package
remote> apt install tmux # install terminal multiplexer to keep sessions after disconnecting (optional)
remote> apt install emacs # install my favourite editor (optional)
remote> adduser xmppusr # add user, so you do not have to run as root
remote> sudo usermod -aG sudo xmppusr # add xmppusr to superuser group
remote> exit # exit, user configs will updated with new login
The following installation steps can be performed by logging into the server using the server IP address from the terminal of your local Linux laptop, which is more convenient (eg keyboard layout).
local> ssh xmppusr@93.130.148.71
local> ssh xmppusr@ddhub.im # optional: updating the DNS entry may take some time (ie hours)
5 Firewall and NGINX server
5.1 Installation
install Firewall (Alternatively, the firwall can also be configured in the HETZNER website):
remote> sudo emacs # optional: work in emacs working environment, start shell with M-x shell
remote> sudo apt install ufw
Install NGINX web server:
remote> sudo apt install nginx certbot python3-certbot-nginx # install NGINX and Let's encrypt certification tool
5.2 Configuration
Configure Firewall:
remote> # Configure Firewall
remote> sudo ufw allow ssh #IMPORTANT! Allow ssh connections before starting firewall, otherwise you will be disconnected!
remote> sudo ufw allow 80/tcp; sudo ufw allow 80/udp # http
remote> sudo ufw allow 443/tcp; sudo ufw allow 443/udp #https
remote> sudo ufw allow 5222/tcp; sudo ufw allow 5222/udp # PROSODY
remote> sudo ufw allow 5269/tcp; sudo ufw allow 5269/udp # PROSODY
remote> sudo ufw allow 8080/tcp; sudo ufw allow 8080/udp # MOVIM
remote> # sudo ufw allow Turnserver # TURN (coturn) # works only after installing TURN (coturn), skip
remote> sudo ufw allow 3478/tcp; sudo ufw allow 3478/udp # TURN (coturn)
remote> sudo ufw allow 5349/tcp; sudo ufw allow 5349/udp; # TURN (coturn)
remote> sudo ufw allow 49152:65535/tcp; sudo ufw allow 49152:65535/udp # TURN (coturn), for clients
remote> sudo ufw enable
Configure NGINX web server:
remote> sudo certbot --nginx --cert-name ddhub.im -d ddhub.im # create certificates
remote> sudo certbot certificates # check certificates
remote> sudo nginx -t # check integrity of config file
remote> sudo systemctl reload nginx # reload nginx configs
6. PROSODY XMPP server
6.1 Installation
Follow these step to install the PROSODY server [see also 10]:
remote> sudo apt-cache search prosody #find prosody package
remote> sudo apt policy prosody # check prosody version, compare with latest version at https://prosody.im/download/
remote> sudo apt install prosody # install prosody
remote> sudo systemctl status prosody # check whether prosody is running (usually autostarted)
remote> sudo systemctl stop prosody # stop prosody because not configured yet
remote> sudo apt install liblua5.4-dev # install LUA language for loading prosody modules (extensions)
remote> sudo apt install lua-unbound
remote> sudo apt install luarocks
remote> sudo prosodyctl install --server=https://modules.prosody.im/rocks/ mod_listusers # optional: test whether prosody modules can be installed
6.2 Configuration
Follow these steps to configure the PROSODY server (see also [11]). The configuration files (/etc/prosody/prosody.conf.lua and /etc/prosody/conf.avail/ddhub.im.conf.lua) should be updated after making a backup copy of the original files. The configs for the ddhub.im server are available at GITHUB (see above). These config files should be updated in places where passwords are defined (see <SetPasswordHere>) and where edits by the admin where made (see ” — EDITED”).
After one has created a file for one’s own domain in /etc/prosody/conf.avail/<mydomain>.conf.lua the file needs to be linked as follows:
remote> ln -s /etc/prosody/conf.avail/ddhub.im.cfg.lua /etc/prosody/conf.d/ddhub.im.cfg.lua # create softlink to available Virtual Host
Create Admin user:
remote> sudo prosodyctl adduser dandelionhub@ddhub.im # Add user with Prosody Command line tool
Copy certificates for domain and create hook for auto-renewal:
remote> certbot --deploy-hook "prosodyctl --root cert import /etc/letsencrypt/live/" # copy certificates in prosody folder and add hook to copy on renewal (omitted flags "renew --force-renewal" to avoid CA rate limits) remote> less /etc/letsencrypt/renewal/ddhub.im.conf # check added hook
Start Prosody Server:
remote> sudo systemctl restart prosody # restart prosody server with new configs (reload is not sufficient to load new configs)
remote> sudo prosodyctl shell user list ddhub.im # list all users (optional)
7. TURN server (coturn) [OPTIONAL]
Note: Audio/Video calls in MOVIM are mostly based on WebRTC, which is enabled in most browser per default. One only needs server support via STUN/TURN if there are connection problems because of NAT (Network address translation) on a private server behind a network router. Therefore, you may not need to set up the TURN (coturn) server. To make calls via MOVIM you have to ADD the other user to contacts and SUBSCRIBE to the other contact (Check notifications!).
7.1 Installation
remote> sudo apt install coturn # adds audio call functionality remote> sudo systemctl status coturn remote> sudo systemctl stop coturn # stop because not configured yet
7.2 Configuration
7.2.1. TURN server
More details on TURN configuration are available [14, 15, 16]:
Note: The config lines must NOT end with a comment, eg realm=turn.mydomain.com # This is a comment, because TURN will give errors.
Update /etc/default/coturn:
TURNSERVER_ENABLED=1
Update /etc/turnserver.conf:
listening-port=3478
tls-listening-port=5349
# set verbose only for debugging, may fill up storage quickly
# verbose
use-auth-secret o
static-auth-secret=SetPasswordHere
realm=turn.ddhub.im
cert=/etc/coturn/certs/cert.pem
pkey=/etc/coturn/certs/privkey.pem
log-file=/var/log/turn.log
cli-password=SetPasswordHere
Add TLS certificates (turnserver has not permissions to /etc/letsencrypt/.):
remote> # sudo fuser -k 80/tcp; sudo fuser -k 443/tcp; # Optional: you may have to kill process bound to ports 80 (http) and 443 (https) first.
remote> sudo certbot certonly --standalone --preferred-challenges http --deploy-hook "systemctl restart coturn" -d turn.ddhub.im
remote> mkdir -p /etc/coturn/certs/
remote> sudo cp -rLT /etc/letsencrypt/live/turn.ddhub.im/ /etc/coturn/certs/
remote> sudo chown -R turnserver:turnserver /etc/coturn/
remote> sudo chmod -R ug+rwx /etc/coturn/
Add Autorenewal of TLS certificates (script in /etc/coturn/renew_certs.sh, see Github files):
remote> chmod u+rwx /etc/coturn/renew_certs.sh
remote> sudo certbot --deploy-hook "/etc/coturn/renew_certs.sh" # copy certificates in coturn folder and add hook to copy on renewal (omitted flags "renew --force-renewal" to avoid CA rate limits)
remote> less /etc/letsencrypt/renewal/{ddhub.im.conf,turn.ddhub.im.conf} # check certbot hook, go to next file with ":n"
Start Turnserver (coturn):
remote> sudo systemctl start coturn
7.2.1 PROSODY configuration
Update PROSODY config /etc/prosody/prosody.conf.lua:
"turn_external"; -- Provide external STUN/TURN service for e.g. audio/video calls
turn_external_host = "turn.ddhub.im"
turn_external_port = 3478
turn_external_secret = "SetPasswordHere"
Restart PROSODY:
remote> sudo systemctl restart prosody
8. MOVIM
The current recommendations for the installation and configuration of MOVIM are available [12].
Notes:
- To activate OMEMO encryption in MOVIM activate OMEMO in MOVIM > Configuration > Configuration > Use OMEMO. You also have to activate OMEMO in each chat by clicking on the little “Lock” symbol next to the messaging window.
8.1 Installation of system dependencies
remote> sudo apt install git gh # install git and github tools
remote> sudo apt install composer php-fpm php-curl php-mbstring php-imagick php-gd php-pgsql php-xml
remote> sudo apt install postgresql postgresql-contrib # install postresql
remote> sudo systemctl enable postgresql # enable postgresql
remote> sudo -i -u postgres createuser movimusr # create sql user
remote> sudo -i -u postgres createdb -O movimusr movimdb # create sql database
remote> sudo -i -u postgres psql
postgres> \list
postgres> \password movimusr # enter <yoursecretpassword> avoid special characters like # because you will need to add it to the .env config file later
postgres> \quit
8.2 Installation of MOVIM
remote> su - # change role to superuser
remote> cd /var/www/
remote> mkdir ./movim/; chown www-data:www-data ./movim/; cd ./movim/
remote> sudo -u www-data git clone https://github.com/movim/movim.git .
remote> sudo -u www-data git git checkout tags/v0.28 # This command uses a specific TAG, since the latest HEAD may not be completely stable.
remote> git config advice.detached head false # This command ignores warnings about a "detached head" state because you did not checkout the latest commit in the branch but the one tagged as v0.28.
8.3 Configuration of MOVIM
remote> # Create Directories and set user, group and permission (Note: www-data is the default user for NGINX web server, see nginx.conf)
remote> sudo -u www-data mkdir /var/www/movim/cache/; sudo -u www-data mkdir /var/www/movim/log/; sudo -u www-data mkdir /var/www/movim/public/log/
remote> cd /var/www/movim/
remote> sudo -u www-data composer install # ignore superuser warning, will update user, group and permission later
remote> sudo -u www-data cp .env.example .env
remote> sudo -u www-data nano .env # update to: DB_DATABASE=movimdb, DB_USERNAME=movimusr, DB_PASSWORD=<yoursecretpassword>, DAEMON_URL=https://ddhub.im
remote> sudo -u www-data composer movim:migrate # ignore superuser warning, will update user, group and permission later
8.4 Configuration of NGINX for MOVIM
Update the NGINX configs for MOVIM (update /etc/nginx/nginx.conf and create /etc/nginx/conf.d/ddhub.im.conf) are available [Download].
remote> sudo systemctl reload nginx
8.5 Configuration of PHP for MOVIM
Update /etc/php/8.2/cli/php.ini as follows:
opcache.enable=1
opcache.enable_cli=1
Update file /etc/php/8.2/fpm/pool.d/www.conf as follows (otherwise you will get a 502 error when accessing https:/ddhub.im in the browser) (see [13]) :
listen.allowed_clients = 127.0.0.1
; listen = /run/php/php8.2-fpm.sock
listen = 9000
8.6 Configuration of TURNSERVER for MOVIM
Activate Audio and Video in MOVIM user interface > Configuration > Video & Voice > Click “Audio/Preview” (Accept Popup) & “Video/Preview” (Accept Popup)
8.7 Start MOVIM
Restart services:
remote> sudo systemctl | grep php
remote> sudo systemctl reload php8.2-fpm
remote> sudo systemctl reload nginx
Start MOVIM demon as a background process 1) via tmux (terminal multiplexer) [simple] or 2) via systemd [advanced]:
1) To start MOVIM demon via tmux in a permanent terminal (which will persist even if you log off):
remote> sudo apt install tmux # install terminal multiplexer to run MOVIM in the background after logging off
remote> tmux # use "tmux attach" to reconnect to the same tmux session
remote> cd /var/www/movim
remote> sudo -u www-data php daemon.php start # Launch the MOVIM daemon
2) To start MOVIM demon via the systemd tools (see [25]):
remote> sudo cp /var/www/movim/etc/systemd/system/movim.service /etc/systemd/system/movim.service # copy service configuration template into system folder with service files, update according to your system settings, here no update required
remote> sudo systemctl daemon-reload # make systemd reload all service files, incl. movim.service, to register it.
remote> sudo systemctl enable movim.service # register MOVIM with systemd services
remote> sudo systemctl start movim.service # start MOVIM demon
remote> sudo systemctl status movim.service # check MOVIM status
remote> sudo journalctl -xeu movim.service # check MOVIM log
To stop the Movim Demon:
remote> sudo systemctl stop movim.service
remote> sudo systemctl disable movim.service
Note: You may also try as a third option `php daemon.php start &` (ie ending command with an ampersand), but this will make managing the process (incl debugging) more difficult than the other options.
9. Login to MOVIM
9.1 Register Admin account
Go to https://ddhub.im and login with your admin JID, here: dandelionhub@ddhub.im
Set XMPP user (JID) as PHP admin after you have logged in at least once via the MOVIM ddhub.im website:
remote> cd /var/www/movim
remote> sudo -u www-data php daemon.php setAdmin dandelionhub@ddhub.im
You can remove an admin user again:
remote> cd /var/www/movim
remote> sudo -u www-data php php daemon.php setAdmin --remove noadminusr@ddhub.im
remote> sudo php daemon.php help setAdmin # more info on setAdmin command
9.2 Register Domain in MOVIM network
When you click “Sign Up” at the landing page (http://ddhub.im) only the MOV.IM network will be shown. To show also the DHUB.IM domain to sign up an account, you have to first enter all details in the “Administration” (!) Panel. The info in the “Configuration” Panel is not sufficient.
Check here: https://ddhub.im/infos
9.3 Register rooms on Jabber (XMPP) network
If you have created a room (channel, topic) and want others on the XMPP (aka Jabber) network to find it, you have to create a TYPE A record for <rooms.ddhub.im>, wait a few hours till the record is propagated globally, and then invite <crawler@search.jabber.network> to your room [20, 21, 22, 23].
Note: Rooms with one or less occupants will not be shown in the frontend on searches of search.jabber.network (to combat a specific kind of room spam abuse). I overcame this problem by creating another user <roomsbot@ddhub.im>, which I used to joing the room immediately after I had created a room withanother user. The room showed in the search using the search.jabber.network.
10 Update/Upgrade/Reinstall MOVIM (Optional)
The development cycles of MOVIM are short with a release every month, so you may want to upgrade a MOVIM installation every 3 months or less. An upgrade of other components of your software stack is usually not required.
Note: Differences in sequence of commands are marked bold.
Note: Code in 10.1 and 10.2 has been changed since last update, needs retesting.
10.1 Option 1: Update/Upgrade MOVIM
In principle, a MOVIM upgrade only requires an update of the code in the /var/www/movim/ folder. The recommended instructions are in INSTALL.md > #Update [26]:
remote> sudo systemctl stop movim.service # stop current movim service
remote> cd /var/www/movim/ # change to movim folder
remote> cp -r /var/www/movim /var/www/movim_v25.1_bkp20241102_0936 # backup movim folder, check if backup worked!
remote> sudo -u www-data git pull # update the Movim source-code assuming the present working directory is a valid local git repository
remote> sudo -u www-data git checkout tags/v0.28 # This command uses a specific TAG, since the latest HEAD may not be completely stable. Use latest, stable Movim version (check on Github repo under "tags")
remote> sudo -u www-data composer install
remote> sudo -u www-data composer movim:migrate
remote> sudo systemctl start movim.service #start upgraded movim service
10.2 Option 2: Reinstall MOVIM
If the instructions in section 10.1 do not work, eg because of issues with git and complex code merges, one can install an upgraded MOVIM version from scratch following sections 8.2 and 8.3 with the required steps summarized below:
remote> sudo systemctl stop movim.service # stop current movim service
remote> cd /var/www/movim/ # change to movim folder
remote> cp -r /var/www/movim /var/www/movim_v25.1_bkp20241102_0936 # backup movim folder, check if backup worked!
remote> rm -r /var/www/movim/* # delete current movim files
remote> sudo -u www-data git clone https://github.com/movim/movim.git .
remote> sudo -u www-data git checkout tags/v0.28 # This command uses a specific TAG, since the latest HEAD may not be completely stable. Use latest, stable Movim version (check on Github repo under "tags")
remote> sudo -u www-data mkdir /var/www/movim/cache/; sudo -u www-data mkdir /var/www/movim/log/; sudo -u www-data mkdir /var/www/movim/public/log/
remote> sudo -u www-data cp /var/www/movim_v25.1_bkp20241102_0936/.env /var/www/movim/.env # copy configs from previous movim installation
remote> sudo -u www-data composer install
remote> sudo -u www-data composer movim:migrate
remote> sudo systemctl start movim.service #start upgraded movim service
Note: I could successfully restart Movim again, but systemctl status movim.service showed errors (“movim.ERROR: unable to open image”). It seems the PostGres Database used by Movim is linking files in the /var/www/movim/cache/ directory and Movim gives errors if they are not found in the new installation. However, stopping and restarting Movim service, fixed this problem for me.
11. Debugging
11.1 Check PROSODY
remote> sudo systemctl restart prosody
remote> sudo prosodyctl check
remote> sudo prosodyctl check connectivity
remote> sudo prosodyctl check turn
remote> less /var/log/prosody/prosody.err
remote> less /var/log/prosody/prosody.log
11.2 Check NGINX
remote> sudo systemctl restart nginx
remote> sudo nginx -t
remote> sudo journalctl -xeu nginx
remote> sudo less /var/log/nginx/error.log
remote> sudo less /var/log/letsencrypt/letsencrypt.log
I have come across this error “nginx[69307]: nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)”, which needs killing binding of nginx and processes:
remote> sudo fuser -k 443/tcp; # kills process bound to port, do likewise for all other port addresses in use
Th error “An unexpected error occurred: AttributeError: can’t set attribute” may be caused by having create a certificate too many times in a time interval (see less /var/log/letsencrypt/letsencrypt.log).
11.3 Check MOVIM
remote> cd /var/www/movim/; sudo -u www-data php daemon.php start # alternative run shell as www-data: sudo -s -u www-data;
remote> less /var/www/movim/log/errors.log
11.4 Delete a user
If you want to permanently delete a user, you have to delete it in the MOVIM and PROSODY databases.
Option 1: The best way is to delete in the MOVIM GUI > Configuration > Account > “Delete my account”.
Option 2a: Connect via an ssh terminal with the MOVIM server and delete the MOVIM in the postgres database.
local> ssh xmppusr@ddhub.im
remote> sudo -i -u postgres psql # connect to postgres
psql> \list # list databases
psql> \connect movimdb
psql> \dt # list tables in database
psql> delete from users where username = 'dandelionhub' and domain = 'ddhub.im' cascade'; # delete user from all tables, watch out the single apostrophe before the semicolon at the end of command
psql> \quit # exit psql terminal
Option 2b: Connect via an ssh terminal
local> ssh xmppusr@ddhub.im
remote> sudo prosodyctl shell user list ddhub.im
remote> sudo prosodyctl deluser dandelionhub@ddhub.im
Note: If the user was an Admin, the information in the Administration panel will be preserved.
Some info about user registrations in the MOVIM/XMPP eco system is available [24].
References
- https://en.wikipedia.org/wiki/XMPP
- https://xmpp.org
- https://prosody.im
- https://movim.eu
- https://mov.im
- http://whatsapp.com
- https://www.hosting.de
- https://www.hetzner.com
- https://www.namecheap.com
- https://prosody.im/download/start
- https://prosody.im/doc/configure
- https://github.com/movim/movim/blob/master/INSTALL.md
- https://stackoverflow.com/questions/21524373/nginx-connect-failed-111-connection-refused-while-connecting-to-upstream
- https://prosody.im/doc/turn
- https://prosody.im/doc/coturn
- https://fatiherikci.com/en/how-to-install-turn-coturn/
- https://mov.im/post/pubsub.movim.eu/Movim/end-to-end-encryption-in-movim-omemo-is-finally-there-yudZPP
- https://slidge.im/core/user/index.html
- https://sr.ht/~nicoco/slidge/
- https://prosody.im/doc/chatrooms
- https://search.jabber.network
- https://search.jabber.network/docs/operators
- https://search.jabber.network/docs/owners
- https://github.com/movim/movim/issues/1360#issuecomment-2337327228
- https://linuxhandbook.com/create-systemd-services/
- https://github.com/movim/movim/blob/master/INSTALL.md#update