Useful Linux CLI commands

hostname -I
[root@ftp ~]# groups
[root@ftp ~]# groups root
[root@ftp ~]# grep root /etc/group

Symbolic Links

Hard Link

ln /path/to/file /path/to/hardlink

Soft Link

ln -s /path/to/file /path/to/link

To Prevent Soft Link Overwrite

ln -s -b /path/to/file /path/to/link

List all symbolic links

ls -lt

Unlink a file

unlink linkfile
grep -r SSLCertificateFile /etc/httpd
httpd -S
openssl x509 -in server.crt -noout -subject

All users:

$ getent passwd

All groups:

$ getent group

All groups with a specific user:

$ getent group | grep user

Delete user

userdel user's username

Delete including users home directory

userdel -r user's username

Run shell script in Apache Airflow

Create the script.py inside the ../airflow/dags folder

from builtins import range
from datetime import timedelta

import airflow
from airflow.models import DAG
from airflow.operators.bash_operator import BashOperator
from airflow.operators.dummy_operator import DummyOperator

args = {
    'owner': 'airflow',
    'start_date': airflow.utils.dates.days_ago(2),
}

dag = DAG(
    dag_id='dagid1',
    default_args=args,
    schedule_interval='0 0 * * *',
    dagrun_timeout=timedelta(minutes=60),
)

# [START howto_operator_bash]
create_command = """
echo 'Hello world';
echo 'Welcome';"""

run_this = BashOperator(
    task_id='taskid1',
    bash_command=create_command,
    dag=dag,
)

Apache Airflow Beginners guide

export AIRFLOW_HOME=~/airflow

# install from pypi using pip
pip install apache-airflow

# initialize the database
airflow initdb

# start the web server, default port is 8080
airflow webserver -p 8080

# start the scheduler
airflow scheduler

# visit localhost:8080 in the browser and magnum pharmaceuticals enable the example dag in the home page

Nginx reverse proxy

Airflow reverse proxy without rewrite

server {
  listen 80;
  server_name lab.example.com;

  location /myorg/airflow/ {
      proxy_pass http://localhost:8080;
      proxy_set_header Host $host;
      proxy_redirect off;
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";
  }
}

Airflow reverse proxy with rewrite

server {
    listen 80;
    server_name lab.example.com;

    location /myorg/workflow/ {
        rewrite ^/myorg/workflow/(.*)$ /$1 break;  # remove prefix from http header
        proxy_pass http://localhost:5555;
        proxy_set_header Host $host;
        proxy_redirect off;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

Apache port to subfolder reverse proxy

Config file location

RHEL based distributions

httpd.conf

Debian

sites-available/default

Updating & Installing

Update package lists:

apt-get update

Install Apache2 with mod_proxy:

sudo apt-get install libapache2-mod-proxy-html

Install libxml2 module:

apt-get install libxml2-dev

On /etc/apache2/apache2.conf add below code

LoadModule  proxy_module         /usr/lib/apache2/modules/mod_proxy.so
LoadModule  proxy_http_module    /usr/lib/apache2/modules/mod_proxy_http.so
LoadModule  headers_module       /usr/lib/apache2/modules/mod_headers.so
LoadModule  deflate_module       /usr/lib/apache2/modules/mod_deflate.so
LoadFile    /usr/lib/x86_64-linux-gnu/libxml2.so

Method 1

Add a sample reverse proxy configuration to the same file /etc/apache2/apache2.conf

ProxyPass /subfolder http://127.0.0.1:9999/
ProxyPassReverse /subfolder http://127.0.0.1:9999/

Restart Apache2:

systemctl restart apache2

Method 2

Config file

<VirtualHost *:80>

        ServerName example.com
        ServerAlias www.example.com
        ServerAdmin example@example.com

       <Location /app1>
             ProxyPass http://localhost:4000/
      </Location>

       <Location /app2>
             ProxyPass http://localhost:4001/
      </Location>

</VirtualHost>