nihilist - 16 / 07 / 2023

wikiless Setup

In this tutorial we're going to check out how to install Wikiless, a privacy front-end for wikipedia.

Initial Setup

First git clone the repository

[ Datura Network ] [ /dev/pts/2 ] [/srv]
→ git clone https://github.com/Metastem/wikiless
Cloning into 'wikiless'...
remote: Enumerating objects: 1080, done.
remote: Counting objects: 100% (314/314), done.
remote: Compressing objects: 100% (135/135), done.
remote: Total 1080 (delta 216), reused 250 (delta 175), pack-reused 766
Receiving objects: 100% (1080/1080), 488.53 KiB | 8.14 MiB/s, done.
Resolving deltas: 100% (598/598), done.

[ Datura Network ] [ /dev/pts/2 ] [/srv]
→ cd wikiless
	

run the docker files

[ Datura Network ] [ /dev/pts/0 ] [/srv/wikiless]
→ git pull
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 6 (delta 4), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (6/6), 1.32 KiB | 1.32 MiB/s, done.
From https://github.com/Metastem/wikiless
   cd561d0..23087c5  main       -> origin/main
Updating cd561d0..23087c5
Fast-forward
 README.md          | 12 ------------
 docker-compose.yml |  8 +++-----
 2 files changed, 3 insertions(+), 17 deletions(-)

[ Datura Network ] [ /dev/pts/0 ] [/srv/wikiless]
→ vim docker-compose.yml ; vim wikiless.config 

[ Datura Network ] [ /dev/pts/0 ] [/srv/wikiless]
→ cat wikiless.config
const config = {
  /**
  * Set these configs below to suite your environment.
  */
  domain: process.env.DOMAIN || 'wikiless.datura.network', // Set to your own domain
  default_lang: process.env.DEFAULT_LANG || 'en', // Set your own language by default
  theme: process.env.THEME || 'dark', // Set to 'white' or 'dark' by default
  http_addr: process.env.HTTP_ADDR || '0.0.0.0', // don't touch, unless you know what your doing
  nonssl_port: process.env.NONSSL_PORT || 8080, // don't touch, unless you know what your doing

  /**
  * You can configure redis below if needed.
  * By default Wikiless uses 'redis://127.0.0.1:6379' as the Redis URL.
  * Versions before 0.1.1 Wikiless used redis_host and redis_port properties,
  * but they are not supported anymore.
  * process.env.REDIS_HOST is still here for backwards compatibility.
  */
  redis_url: process.env.REDIS_URL || process.env.REDIS_HOST || 'redis://127.0.0.1:6379',
  redis_password: process.env.REDIS_PASSWORD,

  /**
  * You might need to change these configs below if you host through a reverse
  * proxy like nginx.
  */
  trust_proxy: process.env.TRUST_PROXY === 'true' || true,
  trust_proxy_address: process.env.TRUST_PROXY_ADDRESS || '127.0.0.1',

  /**
  * Redis cache expiration values (in seconds).
  * When the cache expires, new content is fetched from Wikipedia (when the
  * given URL is revisited).
  */
  setexs: {
    wikipage: process.env.WIKIPAGE_CACHE_EXPIRATION || (60 * 60 * 1), // 1 hour
  },

  /**
  * Wikimedia requires a HTTP User-agent header for all Wikimedia related
  * requests. It's a good idea to change this to something unique.
  * Read more: https://meta.wikimedia.org/wiki/User-Agent_policy
  */
  wikimedia_useragent: process.env.wikimedia_useragent || 'Wikiless media proxy bot (https://github.com/Metastem/wikiless)',

  /**
  * Cache control. Wikiless can automatically remove the cached media files from
  * the server. Cache control is on by default.
  * 'cache_control_interval' sets the interval for often the cache directory
  * is emptied (in hours). Default is every 24 hours.
  */
  cache_control: process.env.CACHE_CONTROL !== 'true' || true,
  cache_control_interval: process.env.CACHE_CONTROL_INTERVAL || 24,
}

module.exports = config

[ Datura Network ] [ /dev/pts/0 ] [/srv/wikiless]
→ cat docker-compose.yml
version: "3.7"

services:
  wikiless:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: wikiless
    hostname: wikiless
    restart: always
    networks:
      wikiless_net:
        ipv4_address: 172.4.0.6
    environment:
      REDIS_HOST: redis://172.4.0.5:6379
    ports:
      - "127.0.0.1:8180:8080" # change port if needed
    security_opt:
      - no-new-privileges:true
    cap_drop:
      - ALL
    depends_on:
        - wikiless-redis

  wikiless-redis:
    container_name: wikiless-redis
    hostname: wikiless-redis
    image: redis:latest
    restart: always
    networks:
      wikiless_net:
        ipv4_address: 172.4.0.5
    ports:
      - "6379"
    user: nobody
    read_only: true
    security_opt:
      - no-new-privileges:true
    tmpfs:
      - /data:size=10M,mode=0770,uid=65534,gid=65534,noexec,nosuid,nodev
    cap_drop:
      - ALL
    cap_add:
      - SETGID
      - SETUID
      - DAC_OVERRIDE

networks:
  wikiless_net:
    ipam:
      config:
        - subnet: 172.4.0.0/16

[ Datura Network ] [ /dev/pts/0 ] [/srv/wikiless]
→ ls
docker-compose.yml  LICENSE.md  media  nginx.conf  package.json  package-lock.json  README.md  SECURITY.md  src  static  wikiless.config

[ Datura Network ] [ /dev/pts/0 ] [/srv/wikiless]
→

[ Datura Network ] [ /dev/pts/0 ] [/srv/wikiless]
→ vim Dockerfile

[ Datura Network ] [ /dev/pts/0 ] [/srv/wikiless]
→ docker-compose down
Stopping wikiless       ... done
Stopping wikiless-redis ... done
Removing wikiless       ... done
Removing wikiless-redis ... done
Removing network wikiless_wikiless_net

[ Datura Network ] [ /dev/pts/0 ] [/srv/wikiless]
→ docker-compose up -d --build
Creating network "wikiless_wikiless_net" with the default driver
Building wikiless
Step 1/9 : FROM node:20-alpine3.17 AS build
20-alpine3.17: Pulling from library/node
4db1b89c0bd1: Pull complete
c14d172ed001: Pull complete
c07dc96d4e30: Pull complete
db1d0c17eb17: Pull complete
Digest: sha256:e6df1a7e4da3c01fee080bfb504dc5b980a19bea23bd1884629469b55d6cd02f
Status: Downloaded newer image for node:20-alpine3.17
 ---> 063cc1778b5d
Step 2/9 : WORKDIR /wikiless
 ---> Running in 85a222226267
Removing intermediate container 85a222226267
 ---> ec73414a5f0a
Step 3/9 : COPY . /wikiless
 ---> 97c84d369440
Step 4/9 : RUN npm install --no-optional
 ---> Running in 53377f121301
npm WARN config optional Use `--omit=optional` to exclude optional dependencies, or
npm WARN config `--include=optional` to include them.
npm WARN config
npm WARN config     Default value does install optional deps unless otherwise omitted.

added 117 packages, and audited 118 packages in 2s

23 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
npm notice
npm notice New minor version of npm available! 9.7.2 -> 9.8.0
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v9.8.0>
npm notice Run `npm install -g npm@9.8.0` to update!
npm notice
Removing intermediate container 53377f121301
 ---> a0ca7a91b7b4

Step 5/9 : FROM gcr.io/distroless/nodejs20-debian11
latest: Pulling from distroless/nodejs20-debian11
a7ca0d9ba68f: Already exists
fe5ca62666f0: Already exists
b02a7525f878: Already exists
fcb6f6d2c998: Already exists
e8c73c638ae9: Already exists
1e3d9b7d1452: Already exists
4aa0ea1413d3: Already exists
7c881f9ab25e: Already exists
5627a970d25e: Already exists
96266735468f: Already exists
2758d0c31c8c: Already exists
08553ba93cfe: Already exists
dfc02eb7708f: Already exists
52907d314ddc: Already exists
4eec690774a4: Already exists
960f4c0076bf: Already exists
Digest: sha256:9468dc4069714f71a30c6075027f75edca89cc4b30d1afc6741c3430def76d7f
Status: Downloaded newer image for gcr.io/distroless/nodejs20-debian11:latest
 ---> b0a23627a0ab
Step 6/9 : COPY --from=build /wikiless /wikiless
 ---> 72b7dee566ed
Step 7/9 : WORKDIR /wikiless
 ---> Running in 3f227aaf0a85
Removing intermediate container 3f227aaf0a85
 ---> 1cb668fc638e
Step 8/9 : COPY wikiless.config config.js
 ---> cba58fa6593a
Step 9/9 : CMD ["src/wikiless.js"]
 ---> Running in ac6bf21520f9
Removing intermediate container ac6bf21520f9
 ---> 7c3ecb0313e1

Successfully built 7c3ecb0313e1
Successfully tagged wikiless_wikiless:latest
Creating wikiless-redis ... done
Creating wikiless       ... done

then install the reverse nginx proxyi, by default the app is available on local port 8180:

[ Datura Network ] [ /dev/pts/2 ] [/srv/wikiless]
→ nmap 127.0.0.1 -p 8180
Starting Nmap 7.93 ( https://nmap.org ) at 2023-07-16 16:01 CEST
Nmap scan report for localhost.localdomain (127.0.0.1)
Host is up (0.00010s latency).

PORT     STATE SERVICE
8180/tcp open  unknown

Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds

root@Datura /srv/wikiless # cd /etc/nginx/sites-available/
root@Datura /etc/nginx/sites-available # vim wikiless.datura.network.conf


[ Datura Network ] [ /dev/pts/0 ] [/etc/nginx/sites-available]
→ cat /etc/nginx/sites-available/wikiless.datura.network.conf
server {
  server_name wikiless.datura.network;

    listen 443 ssl;
    listen [::]:443 ssl;
    #http2 on;
    ssl_certificate /etc/acme/certs/wikiless.datura.network/wikiless.datura.network.cer;
    ssl_certificate_key /etc/acme/certs/wikiless.datura.network/wikiless.datura.network.key;

    #ssl_certificate /etc/letsencrypt/live/wikiless.datura.network/fullchain.pem;
    #ssl_certificate_key /etc/letsencrypt/live/wikiless.datura.network/privkey.pem;
    #include /etc/letsencrypt/options-ssl-nginx.conf;
    #ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    add_header strict_sni on;
    add_header strict_sni_header on;
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header Content-Security-Policy upgrade-insecure-requests;
    add_header X-XSS-Protection "1; mode=block";
    add_header X-Content-Type-Options nosniff;
    add_header X-Frame-Options "DENY";
    add_header Clear-Site-Data "cookies";
    add_header Referrer-Policy "no-referrer";
    add_header Permissions-Policy "interest-cohort=(),accelerometer=(),ambient-light-sensor=(),autoplay=(),camera=(),encrypted-media=(),focus-without-user-activation=(),geolocation=(),gyroscope=(),magnetometer=(),microphone=(),midi=(),payment=(),picture-in-picture=(),speaker=(),sync-xhr=(),usb=(),vr=()";
    resolver 1.1.1.1;

    #ssl_trusted_certificate /etc/letsencrypt/live/wikiless.datura.network/chain.pem;
    #ssl_trusted_certificate /etc/acme/certs/wikiless.datura.network/wikiless.datura.network.cer;
    #ssl_stapling on;
    #ssl_stapling_verify on;

    access_log /dev/null;
    error_log  /dev/null;

   location / {
   proxy_set_header X-Forwarded-For $remote_addr;
   proxy_pass http://localhost:8180;
        }
}

server {
  listen 80;
  listen [::]:80;
  server_name wikiless.datura.network;
  return 301 https://wikiless.datura.network$request_uri;
  }


root@Datura /etc/nginx/sites-available # ln -s /etc/nginx/sites-available/wikiless.datura.network.conf  /etc/nginx/sites-enabled/
root@Datura /etc/nginx/sites-available # systemctl stop nginx
root@Datura /etc/nginx/sites-available # acme.sh --issue --standalone -d wikiless.datura.network -k 4096

root@Datura /etc/nginx/sites-available # nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
root@Datura /etc/nginx/sites-available # systemctl restart nginx

then check that your instance is working here:

Next we're going to make sure the website is accessible over tor:

[ Datura Network ] [ /dev/pts/2 ] [/srv]
→ cat /etc/nginx/sites-available/wikiless.datura.network.conf
server {
  server_name wikiless.datura.network;

    listen 443 ssl;
    listen [::]:443 ssl;
    #http2 on;
    ssl_certificate /etc/acme/certs/wikiless.datura.network/wikiless.datura.network.cer;
    ssl_certificate_key /etc/acme/certs/wikiless.datura.network/wikiless.datura.network.key;

    ######## TOR CHANGES ########
    listen 4444 ssl;
    listen [::]:4444 ssl;
    server_name wikiless.daturab6drmkhyeia4ch5gvfc2f3wgo6bhjrv3pz6n7kxmvoznlkq4yd.onion;
    add_header Onion-Location "http://wikiless.daturab6drmkhyeia4ch5gvfc2f3wgo6bhjrv3pz6n7kxmvoznlkq4yd.onion$request_uri" always;
    ######## TOR CHANGES #######

    #ssl_certificate /etc/letsencrypt/live/wikiless.datura.network/fullchain.pem;
    #ssl_certificate_key /etc/letsencrypt/live/wikiless.datura.network/privkey.pem;
    #include /etc/letsencrypt/options-ssl-nginx.conf;
    #ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    add_header strict_sni on;
    add_header strict_sni_header on;
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header Content-Security-Policy upgrade-insecure-requests;
    add_header X-XSS-Protection "1; mode=block";
    add_header X-Content-Type-Options nosniff;
    add_header X-Frame-Options "DENY";
    add_header Clear-Site-Data "cookies";
    add_header Referrer-Policy "no-referrer";
    add_header Permissions-Policy "interest-cohort=(),accelerometer=(),ambient-light-sensor=(),autoplay=(),camera=(),encrypted-media=(),focus-without-user-activation=(),geolocation=(),gyroscope=(),magnetometer=(),microphone=(),midi=(),payment=(),picture-in-picture=(),speaker=(),sync-xhr=(),usb=(),vr=()";
    resolver 1.1.1.1;

    #ssl_trusted_certificate /etc/letsencrypt/live/wikiless.datura.network/chain.pem;
    #ssl_trusted_certificate /etc/acme/certs/wikiless.datura.network/wikiless.datura.network.cer;
    #ssl_stapling on;
    #ssl_stapling_verify on;

    access_log /dev/null;
    error_log  /dev/null;

   location / {
   proxy_set_header X-Forwarded-For $remote_addr;
   proxy_pass http://localhost:8180;
        }
}

server {
  listen 80;
  listen [::]:80;
  server_name wikiless.datura.network;
  return 301 https://wikiless.datura.network$request_uri;
  }

[ Datura Network ] [ /dev/pts/2 ] [/srv]
→ cat /etc/tor/torrc | grep 444
HiddenServicePort 80 127.0.0.1:4443
HiddenServicePort 443 127.0.0.1:4444
	

It may give an ssl error but at least it serves everything over https over tor. Let me know if you managed to make this work without the need of https.

And lastly let's make it update automatically via a cronjob:

[ Datura Network ] [ /dev/pts/2 ] [/srv]
→ crontab -e
	
@daily docker-compose -f /srv/wikiless/docker-compose.yml stop ; git -C /srv/wikiless/ pull ; docker-compose -f /srv/wikiless/docker-compose.yml up -d --build

[ Datura Network ] [ /dev/pts/2 ] [/srv]
→ cronitor select

Use the arrow keys to navigate: ↓ ↑ → ←
? Select job to run:
✔ docker-compose -f /srv/wikiless/docker-compose.yml stop ; git -C /srv/wikiless/ pull ; docker-compose -f /srv/wikiless/docker-compose.yml up -d --build
----► Running command: docker-compose -f /srv/wikiless/docker-compose.yml stop ; git -C /srv/wikiless/ pull ; docker-compose -f /srv/wikiless/docker-compose.yml up -d --build

Stopping wikiless       ... done
Stopping wikiless-redis ... done
Already up to date.
Building wikiless
Step 1/9 : FROM node:20-alpine3.17 AS build
 ---> 063cc1778b5d
Step 2/9 : WORKDIR /wikiless
 ---> Using cache
 ---> ec73414a5f0a
Step 3/9 : COPY . /wikiless
 ---> Using cache
 ---> 97c84d369440
Step 4/9 : RUN npm install --no-optional
 ---> Using cache
 ---> a0ca7a91b7b4

Step 5/9 : FROM gcr.io/distroless/nodejs20-debian11
 ---> b0a23627a0ab
Step 6/9 : COPY --from=build /wikiless /wikiless
 ---> Using cache
 ---> 72b7dee566ed
Step 7/9 : WORKDIR /wikiless
 ---> Using cache
 ---> 1cb668fc638e
Step 8/9 : COPY wikiless.config config.js
 ---> Using cache
 ---> cba58fa6593a
Step 9/9 : CMD ["src/wikiless.js"]
 ---> Using cache
 ---> 7c3ecb0313e1

Successfully built 7c3ecb0313e1
Successfully tagged wikiless_wikiless:latest
Starting wikiless-redis ... done
Recreating wikiless     ... done

----► ✔ Command successful    Elapsed time 13.038s