Tools

MongoDB - Tutorial

Good tutorial video https://www.youtube.com/watch?v=pWbMrx5rVBE Handson Install on Ubuntu sudo su wget -qO - https://www.mongodb.org/static/pgp/server-4.4.asc | sudo apt-key add - echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/4.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.4.list apt update sudo apt install -y mongodb-org systemctl start mongodb Playground in local Docker container docker run -p 27017:27017 -d mongo:4.0-xenial Toolkit curl https://fastdl.mongodb.org/tools/db/mongodb-database-tools-ubuntu2004-x86_64-100.3.0.deb -O sudo apt install ./mongodb-database-tools-ubuntu2004-x86_64-100.3.0.deb Mongo shell mkdir mongosh cd mongosh curl https://downloads.mongodb.com/compass/mongosh-0.8.3-linux.tgz -O tar -zxvf mongosh-0.

GitLab on Docker

Install GitLab running on Docker Here is the official document. I followed it. Caution: I’m using my GitLab on VM, not Docker image. Because the official image doesn’t contain e-mail client, and it is a little bit hard to understand. For me, deploying it on premise or VM is easy. Install docker apt install -y docker.io Set path export GITLAB_HOME=/srv/gitlab Configurations and repositories would be stored in this path. Run a container sudo docker run --detach \ --hostname {{ your_hostname }} \ --publish 443:443 --publish 80:80 --publish 2222:22 \ --name gitlab \ --restart always \ --volume $GITLAB_HOME/config:/etc/gitlab \ --volume $GITLAB_HOME/logs:/var/log/gitlab \ --volume $GITLAB_HOME/data:/var/opt/gitlab \ gitlab/gitlab-ce:latest I changed git port to 2222 because the server should be access by ssh.

SolrCloud - set up

Install Solr Install environment Ubuntu 20.04 Pre-installation Linux settings Create an user solr. sudo useradd -r -s /bin/bash solr Change several kernel parameters in /etc/security/limits.conf. solr hard nofile 65535 solr soft nofile 65535 solr hard nproc 65535 solr soft nproc 65535 Turn off swaps. sudo swapoff -a Relogin so that the change will takes a place. Install Java Install Java. sudo apt install -y openjdk-11-jdk Install ZooKeeper Install Apache ZooKeeper.

Setting up Apache ZooKeeper cluster

What is ZooKeeper (very rough) Apache ZooKeeper storage hierarchal structure. Refer to ZNode. https://de.slideshare.net/sauravhaloi/introduction-to-apache-zookeeper Install ZooKeeper My environment Ubuntu 20.04 Pre-installation Install java first. sudo apt update sudo apt upgrade -y sudo apt install -y openjdk-11-jdk Crete ZooKeeper user. sudo useradd -r -s /bin/bash zk Install ZooKeeper on nodes Download link https://zookeeper.apache.org/releases.html sudo su cd /opt curl https://mirror.netcologne.de/apache.org/zookeeper/zookeeper-3.6.2/apache-zookeeper-3.6.2-bin.tar.gz -O tar xvf apache-zookeeper-3.6.2-bin.tar.gz mkdir /var/lib/zookeeper cd apache-zookeeper-3.6.2-bin cp conf/zoo_sample.cfg conf/zoo.

lsyncd - synchronize daemon

What’s lsyncd My understanding is, it’s kind of rsync daemon. Install On Ubuntu Install on source servers (no need to install on destination servers). apt update apt install lsyncd Set kernel parameter fs.inotify.max_user_watches. /etc/sysctl.conf # for lsyncd fs.inotify.max_user_watches = 8192000 Before running We should configure firewall so that the source server can SSH to destination servers. Here is my ssh snippets. Use (sync) Configuration Here is the configuration example. mkdir /etc/lsyncd vim /etc/lsyncd/lsyncd.

Set up my Vim

Install jellybeans I use a color scheme called “Jellybeans”. mkdir ~/.vim cd ~/.vim sudo yum install -y git git clone https://github.com/nanotech/jellybeans.vim mkdir colors mv jellybeans.vim/colors/jellybeans.vim ~/.vim/colors/jellybeans.vim vim ~/.vimrc colorscheme jellybeans set expandtab set tabstop=2 set softtabstop=2 set shiftwidth=2 set number syntax on set t_Co=256 set foldmethod=syntax Add the line into ~/.bashrc, TERM=xterm-256color vim plugin manager https://github.com/junegunn/vim-plug Install curl -fLo ~/.vim/autoload/plug.vim --create-dirs \ https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim Configure. https://github.com/junegunn/vim-plug/wiki/tutorial In ~/.vimrc call plug#begin('~/.

Easy search with browser (Lexico exmple)

Vivaldi (or Chromium or Chrome) setting Go to Preferences -> Search. Sample I use Lexico often to search the meaning of words (as dictionary). Name: Lexico Nickname: lex URL: https://www.lexico.com/en/definition/%s Check the Use POST method With this configuration, I can search the word, for example, “test” by type “lex test” in the browser’s URL box.

Jupyter Notebook

Environment macOS: 10.15.3 Python: pyenv installed Install pip install -U pip pip install jupyter How to Run jupyter-notebook run the Jupyter. % jupyter-notebook [I 19:21:07.376 NotebookApp] Serving notebooks from local directory: /Users/atlex/nlp [I 19:21:07.376 NotebookApp] The Jupyter Notebook is running at: [I 19:21:07.376 NotebookApp] http://localhost:8888/?token=8637cf3f5b526b4ac33e93964844ecf7626ccd7be92616a2 [I 19:21:07.376 NotebookApp] or http://127.0.0.1:8888/?token=8637cf3f5b526b4ac33e93964844ecf7626ccd7be92616a2 [I 19:21:07.376 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). [C 19:21:07.