How to Install CKAN for Open Data Portal on Ubuntu 20.04

Comprehensive Knowledge Archive Network (CKAN) is an open source software for open data portal, CMS that can help us manage and publish data sets. CKAN is used by central and local governments, research institutions, and other organizations to publish data.

An example of usage in Indonesia is Satu Data Indonesia (SDI) which is a government data management policy with the aim of creating quality data, easily accessible, and can be shared between central and regional agencies. This SDI policy is contained in Presidential Regulation No. 39 of 2019 concerning One Indonesian Data. Through SDI, all government data and data from other related agencies can end up in the Satu Data Indonesia Portal (data.go.id). Examples of local governments that have open data portals are Open Data Jakarta, Open Data Bandung, and Open Data West Kalimantan.

*Tech Stack*

CKAN uses the Python programming language as the backend, JavaScript as the frontend, the PostgreSQL database and the Solr search engine. CKAN allows for additional extensions or features because it uses a modular architecture. Besides that, CKAN also has an API feature.

*System Requirements*

-Hardware Requirements-

*For portals on a national scale with high traffic:*

· 2 servers with 8GB RAM. One server for web and the other for database/Solr · 160GB storage. The web server can be bigger, as needed, how much data you want to store · Quad core processor. Faster is better

*For regional level portals with lower traffic:*

· 2 servers with RAM 2GB · Storage 80GB · Dual core processor

/Web and database can be combined on the same server, but recommends separating them for security reasons./

-Software Requirements-

*Operating system* Ubuntu 16.04 64bit is recommended, CKAN is available in .deb package. CKAN has also been tested on CentOS and other major Linux distributions.

*Web server*

· Apache · Nginx, for https and caching · Python 2.7.x

*Database server*

· Postgres 9.3+ · Redis · Tomcat/Jetty + Solr

/The system requirements above are based on CKAN’s hosting guidelines, last updated 12-20-2017./

*Specifications in the Tutorial*

Server specifications and software used in this tutorial:

· VPS 1CPU, RAM 2GB, Disk 50GB · Ubuntu 20.04 · Nginx 1.18 · Python 3.8.5 · Supervisor 4.1.0 · PostgreSQL 12.4 · Redis 5.0.7 · Solr 3.6.2 · CKAN package 2.9.1

Port used:

Service Port Used for Nginx 80 Proxy uWSGI 8080 Web server uWSGI 8800 DataPusher Solr/Jetty 8983 Search PostgreSQL 5432 Database Redis 6379 Search

*1. Install CKAN package*

Update Ubuntu.

[INPUT]

1 2 sudo apt update sudo apt upgrade -y

Install package dependencies.

[INPUT]

1 sudo apt install -y libpq5 redis-server nginx supervisor

Download CKAN package.

[INPUT]

1 wget http://packaging.ckan.org/python-ckan_2.9-py3-focal_amd64.deb

Install CKAN package.

[INPUT]

1 sudo dpkg -i python-ckan_2.9-py3-focal_amd64.deb

*2. Install PostgreSQL*

Install PostgreSQL database.

[INPUT]

1 sudo apt install postgresql -y

Check if PostgreSQL is installed properly by displaying the existing database.

[INPUT]

1 sudo -u postgres psql -l

Create a PostgreSQL user named ckan_default.

[INPUT]

1 sudo -u postgres createuser -S -D -R -P ckan_default

Create a PostgreSQL database named ckan_default.

[INPUT]

1 sudo -u postgres createdb -O ckan_default ckan_default -E utf-8

Check whether the ckan_default database was successfully created.

[INPUT]

1 sudo -u postgres psql -l

The result.

[INPUT]

1 2 3 4 5 6 7 8 9 10                                  List of databases      Name     |    Owner     | Encoding | Collate |  Ctype  |   Access privileges   ————–+————–+———-+———+———+———————– ckan_default | ckan_default | UTF8     | C.UTF-8 | C.UTF-8 | postgres     | postgres     | UTF8     | C.UTF-8 | C.UTF-8 | template0    | postgres     | UTF8     | C.UTF-8 | C.UTF-8 | =c/postgres          +               |              |          |         |         | postgres=CTc/postgres template1    | postgres     | UTF8     | C.UTF-8 | C.UTF-8 | =c/postgres          +               |              |          |         |         | postgres=CTc/postgres (4 rows)

*3. Install Solr*

Install Solr.

[INPUT]

1 sudo apt install solr-tomcat -y

Open the Tomcat configuration file.

[INPUT]

1 sudo nano /etc/tomcat9/server.xml

Change the default Tomcat port from 8080 to 8983.

[INPUT]

1 Connector port=”8983″ protocol=”HTTP/1.1″

Replace the default schema.xml file with a symlink to the CKAN schema file included in sources.

[INPUT]

1 2 sudo mv /etc/solr/conf/schema.xml /etc/solr/conf/schema.xml.bak sudo ln -s /usr/lib/ckan/default/src/ckan/ckan/config/solr/schema.xml /etc/solr/conf/schema.xml

Restart Solr by restarting tomcat.

[INPUT]

1 sudo systemctl restart tomcat9

*4. Configure CKAN*

Open the CKAN configuration file.

[INPUT]

1 sudo nano /etc/ckan/default/ckan.ini

Enable the solr_url option by removing the hashtag (#).

[INPUT]

1 solr_url=http://127.0.0.1:8983/solr

The URL for the CKAN site, for example, uses the subdomain data.defnex.com.

[INPUT]

1 ckan.site_url = http://data.defnex.com

Options for connecting to PostgreSQL databases, for example username = ckan_default, password = secretpassword, and database = ckan_default.

[INPUT]

1 sqlalchemy.url = postgresql://ckan_default:secretpassword@localhost/ckan_default

Initialize the CKAN database.

[INPUT]

1 sudo ckan db init

The result.

[INPUT]

1 2 3 4 2020-11-13 07:12:12,813 INFO  [ckan.cli.db] Initialize the Database 2020-11-13 07:12:15,420 INFO  [ckan.model] CKAN database version upgraded: base -> ccd38ad5fced (head) 2020-11-13 07:12:15,420 INFO  [ckan.model] Database initialised Initialising DB: SUCCESS

Restart the Supervisor and check the status.

[INPUT]

1 2 sudo supervisorctl reload sudo supervisorctl status

The result.

[INPUT]

1 2 3 ckan-datapusher:ckan-datapusher-00   RUNNING   pid 2711, uptime :00:21 ckan-uwsgi:ckan-uwsgi-00             RUNNING   pid 2712, uptime :00:21 ckan-worker:ckan-worker-00           RUNNING   pid 2713, uptime :00:21

Restart Nginx.

[INPUT]

1 sudo systemctl restart nginx

*5. Testing*

Test by browsing http://localhost or http://data.defnex.com.

CKAN web frontpage

*6. Install SSL*

Backup the default ckan server block configuration file.

[INPUT]

1 2 cd /etc/nginx/sites-available/ sudo cp ckan ckan.bak

Open the ckan server block configuration file.

[INPUT]

1 sudo nano ckan

Enter server_name and log options.

[INPUT]

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 proxy_cache_path /tmp/nginx_cache levels=1:2 keys_zone=cache:30m max_size=250m; proxy_temp_path /tmp/nginx_proxy 1 2;   server { listen 80; server_name data.defnex.com;       client_max_body_size 100M;     location / {         proxy_pass http://127.0.0.1:8080/;         proxy_set_header Host $host;         proxy_cache cache;         proxy_cache_bypass $cookie_auth_tkt;         proxy_no_cache $cookie_auth_tkt;         proxy_cache_valid 30m;         proxy_cache_key $host$scheme$proxy_host$request_uri;         # In emergency comment out line to force caching         # proxy_ignore_headers X-Accel-Expires Expires Cache-Control;     }   access_log /var/log/nginx/data.defnex.com_access.log; error_log /var/log/nginx/data.defnex.com_error.log warn;     }

Nginx configuration test.

[INPUT]

1 sudo nginx -t

Restart Nginx.

[INPUT]

1 2 sudo systemctl restart nginx sudo systemctl status nginx

Test again to access http://data.defnex.com, if there are no problems after changing the Nginx configuration.

Install certbot for Let’s Encrypt SSL.

[INPUT]

1 2 sudo snap install –classic certbot sudo ln -s /snap/bin/certbot /usr/bin/certbot

Request SSL for data.defnex.com.

[INPUT]

1 sudo certbot –nginx -d data.defnex.com

Test browse https://data.defnex.com.

CKAN with the https protocol

*7. Create a user sysadmin*

Use the CKAN command line interface to create a first-time sysadmin user and also to create test data.

Enabling virtualenv.

[INPUT]

1 2 . /usr/lib/ckan/default/bin/activate cd /usr/lib/ckan/default/src/ckan

Create a new user with the name *musa* and make it a sysadmin.

[INPUT]

1 ckan -c /etc/ckan/default/ckan.ini sysadmin add musa email=trial@musaamin.web.id name=musa

Hasilnya.

[INPUT]

1 2 3 4 5 6 User “musa” not found Create new user: musa? [Y/n]: y Password : Repeat for confirmation: Successfully created user: musa Added musa as sysadmin

Or if you want to make an existing user as sysadmin.

[INPUT]

1 ckan -c /etc/ckan/default/ckan.ini sysadmin add musa

Test the newly created sysadmin user by logging in on the CKAN web.

CKAN dashboard

*8. Use of Resources*

Check the use of VPS resources using htop.

[INPUT]

1 2 sudo apt install htop -y htop

Used VPS Resources

Good luck 🙂

Leave a Reply

Your email address will not be published.

scroll to top