Goku Coding Experience: April 2022

Wednesday 27 April 2022

How to play sound or mp3 file in python?

#pip install pyaudio pydub

from pydub import AudioSegment
from pydub.playback import play

song = AudioSegment.from_mp3('speech_stranger.mp3')
play(song)

Monday 25 April 2022

How to check the server status in ubuntu?

step: how to run it in a crontab?

10 * * * * /bin/bash /home/ninja/Documents/checkVAserverStatus.sh

code:-

---

import configparser
import socket
import json
import psutil
import subprocess
config = configparser.ConfigParser()
config.read("config.ini")
symServer = config['symphony']

data = {
"camera_name":"none",
"event_time":"none",
"event_type":"none",
"image_url":"none",
"video_url":"none",
"camera_id": symServer["camera_id"]
}

def check():
    ram_used = check_ram()
    hddfree_space = check_directory_size("/")
    gpuRun = check_vadocker()

    if not gpuRun:
        data['policy_id'] = symServer["gpu_policy_id"]
        status = sendEvent(data)

    if ram_used > 95:
        data['policy_id'] = symServer["ram_policy_id"]
        status = sendEvent(data)

    if hddfree_space < 10:
        data['policy_id'] = symServer["hddfull_policy_id"]
        status = sendEvent(data)

def check_ram():
    ram_used = psutil.virtual_memory().percent
    return ram_used

def check_directory_size(path):
    hdd = psutil.disk_usage(path)
    free_space = hdd.free / (2**30)
    return free_space

def check_vadocker():
    subprocess1 = subprocess.Popen('docker ps -a -q --filter ancestor=nvcr.io/nvidia/deepstream:6.0-triton --format="{{.ID}}"',shell=True,stdout=subprocess.PIPE)
    subprocess_return = subprocess1.stdout.read()
    if len(subprocess_return)>5:
        print("docker is running")
        return True
    else:
        print("docker is not running")
        return False

def sendEvent(data):
    try:
        strdata = json.dumps(data)
        print(data)

        # Send data
        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        server_address = (symServer["server_ip"], int(symServer["server_port"]))
        sock.connect(server_address)

        try:
            # Send data
            sock.sendall(strdata.encode())

            # Look for the response
            amount_received = 0
            amount_expected = 5
            while amount_received < amount_expected:
                data = sock.recv(100)
                amount_received += len(data)
            # print('alerting send')
            return True
        except Exception as e:
            # print("server_ip: ",e)
            return False
        finally:
            # print('-- closing socket')
            sock.close()
            return True
    except Exception as e:
        # print("server_ip: ",e)
        return False

if __name__=="__main__":
    check()

How to clean the data, image or video inside a folder in ubuntu periodically?

step: how to run it in a crontab?

0 0 * * * /usr/bin/python3 /home/ninja/Documents/cleaner.py

code:-

---

import os
import shutil
import time
from datetime import datetime,timedelta

days = 60

# main function
def main(path):

   # initializing the count
   deleted_folders_count = 0
   deleted_files_count = 0

   # specify the path
   #path = "/home/ninja/Desktop/old/"

   # converting days to seconds
   # time.time() returns current time in seconds
   seconds = time.time() - (days * 24 * 60 * 60)

   # checking whether the file is present in path or not
   if os.path.exists(path):

        # iterating over each and every folder and file in the path
        for root_folder, folders, files in os.walk(path):

            # comparing the days

                        # checking the current directory files
                        for file in files:

                                # file path
                                file_path = os.path.join(root_folder, file)

                                # comparing the days
                                if seconds >= get_file_or_folder_age(file_path):

                                        # invoking the remove_file function

                                        remove_file(file_path)
                                        deleted_files_count += 1 # incrementing count

   print(f"Total folders deleted: {deleted_folders_count}")
   print(f"Total files deleted: {deleted_files_count}")



def remove_folder(path):

   # removing the folder
   if not shutil.rmtree(path):

        # success message
        print(f"{path} is removed successfully")

   else:

        # failure message
        print(f"Unable to delete the {path}")

def remove_file(path):

   # removing the file
   if not os.remove(path):

        # success message
        print(f"{path} is removed successfully")

   else:

        # failure message
        print(f"Unable to delete the {path}")

def get_file_or_folder_age(path):

   # getting ctime of the file/folder
   # time will be in seconds
   ctime = os.stat(path).st_ctime

   # returning the time
   return ctime

if __name__ == '__main__':

   main("/media/ninja/output/videos/")

   #main("/media/ninja/output/VA_logs/")

How to backup the disk from one drive to another drive in ubuntu?

step: how to run it in a crontab?

3 0 * * * /bin/bash /home/peseyes/Documents/main_rsync.sh

code:-

rsync -avzh --delete /media/ninja/output/ /media/ninja/backup

How to monitor the RAM and kill a docker program?

step: how to run it in crontab?

@reboot sleep 900 && /usr/bin/python3 /home/ninja/Documents/watchdog.py

code:-

---

import psutil
import platform
from datetime import datetime
import os
from datetime import timedelta
import time

def get_size(bytes, suffix="B"):
    """
    Scale bytes to its proper format
    e.g:
        1253656 => '1.20MB'
        1253656678 => '1.17GB'
    """
    factor = 1024
    for unit in ["", "K", "M", "G", "T", "P"]:
        if bytes < factor:
            return f"{bytes:.2f}{unit}{suffix}"
        bytes /= factor

while True:

    svmem = psutil.virtual_memory()
    cVal = svmem.percent
    if cVal > 90:
        print("start killing")
        flag = True

        try:
            os.system('docker rm $(docker stop $(docker ps -a -q --filter ancestor=docker pull nvcr.io/nvidia/deepstream:6.0-triton --format="{{.ID}}"))')
        except:
            flag = False
            print("Fail to kill")

        time.sleep(10)

        svmem = psutil.virtual_memory()
        cVal = svmem.percent

        if flag and cVal < 25:
            try:
                print("restarting")
                os.system("/home/ninja/Documents/run1.sh &")
                os.system("/home/ninja/Documents/run2.sh &")

            except:
                print("Fail to relaunch")

    time.sleep(60)

How to log the CPU, RAM, GPU of a Ubuntu PC?

step: how to run it?

python log_gpu_cpu_stats.py --date-local filename.csv --loop 5

code:-

from __future__ import print_function

import datetime
import psutil
import subprocess
import sys
import time

import contextlib

@contextlib.contextmanager
def smart_open(filename=None, mode='a'):
    """
    Context manager which can handle writing to stdout as well as files.

    If filename is None, the handle yielded is to stdout. If writing to
    stdout, the connection is not closed when leaving the context.
    """
    if filename:
        hf = open(filename, mode)
    else:
        hf = sys.stdout

    try:
        yield hf
    finally:
        if hf is not sys.stdout:
            hf.close()

def check_nvidia_smi():
    try:
        subprocess.check_output(['nvidia-smi'])
    except FileNotFoundError:
        raise EnvironmentError('The nvidia-smi command could not be found.')

check_nvidia_smi()

class Logger():
    def __init__(
            self,
            fname=None, style=None, date_format=None,
            refresh_interval=1, iter_limit=None,
            show_header=True, header_only_once=True,
            show_units=True, sep=',',
            ):

        self.fname = fname if fname else None
        if style is not None:
            self.style = style
        elif self.fname is None:
            self.style = 'tabular'
        else:
            self.style = 'csv'
        self.date_format = date_format
        self.refresh_interval = refresh_interval
        self.iter_limit = iter_limit
        self.show_header = show_header
        self.header_only_once = header_only_once
        self.header_count = 0
        self.show_units = show_units
        self.sep = sep
        self.col_width = 10
        self.time_field_width = 15 if self.date_format is None else \
            max(self.col_width, len(time.strftime(self.date_format)))

        if self.date_format is None:
            self.time_field_name = 'Timestamp' + \
                (' (s)' if self.show_units else '')
        else:
            self.time_field_name = 'Time'

        self.cpu_field_names = [
            'CPU' + (' (%)' if self.show_units else ''),
            'RAM' + (' (%)' if self.show_units else ''),
            'Swap' + (' (%)' if self.show_units else ''),
        ]
        self.gpu_field_names = [
            'GPU' + (' (%)' if self.show_units else ''),
            'Mem' + (' (%)' if self.show_units else ''),
            'Temp' + (' (C)' if self.show_units else ''),
        ]
        self.gpu_queries = [
            'utilization.gpu',
            'utilization.memory',
            'temperature.gpu',
        ]
        self.gpu_query = ','.join(self.gpu_queries)

        self.gpu_names = self.get_gpu_names()

    def get_gpu_names(self):
        res = subprocess.check_output(['nvidia-smi', '-L'])
        return [i_res for i_res in res.decode().split('\n') if i_res != '']

    @property
    def tabular_format(self):
        fmt = '{:>' + str(self.time_field_width) + '} |'
        fmt += ('|{:>' + str(self.col_width) + '} ') \
            * len(self.cpu_field_names)
        for i_gpu in range(len(self.gpu_names)):
            fmt += '|'
            fmt += ('|{:>' + str(self.col_width) + '} ') \
                * len(self.gpu_field_names)
        return fmt

    def write_header_csv(self):
        """
        Write header in CSV format.
        """
        with smart_open(self.fname, 'a') as hf:
            print(self.time_field_name + self.sep, end='', file=hf)
            print(*self.cpu_field_names, sep=self.sep, end='', file=hf)
            for i_gpu in range(len(self.gpu_names)):
                print(self.sep, end='', file=hf)
                print(*['{}:{}'.format(i_gpu, fn)
                        for fn in self.gpu_field_names],
                      sep=self.sep, end='', file=hf)
            print("\n", end='', file=hf) # add a newline

    def write_header_tabular(self):
        """
        Write header in tabular format.
        """
        with smart_open(self.fname, 'a') as hf:
            cols = [self.time_field_name]
            cols += self.cpu_field_names
            for i_gpu in range(len(self.gpu_names)):
                cols += ['{}:{}'.format(i_gpu, fn)
                         for fn in self.gpu_field_names]

            print(self.tabular_format.format(*cols), file=hf)

            # Add separating line
            print('-' * (self.time_field_width + 1), end='', file=hf)
            print(
                '+',
                ('+' + '-' * (self.col_width + 1)) \
                    * len(self.cpu_field_names),
                sep='', end='', file=hf)
            for i_gpu in range(len(self.gpu_names)):
                print(
                    '+',
                    ('+' + '-' * (self.col_width + 1)) \
                        * len(self.gpu_field_names),
                    sep='', end='', file=hf)
            print("\n", end='', file=hf) # add a newline

    def write_header(self):
        if self.style == 'csv':
            self.write_header_csv()
        elif self.style == 'tabular':
            self.write_header_tabular()
        else:
            raise ValueError('Unrecognised style: {}'.format(self.style))
        self.header_count += 1

    def poll_cpu(self):
        """
        Fetch current CPU, RAM and Swap utilisation

        Returns
        -------
        float
            CPU utilisation (percentage)
        float
            RAM utilisation (percentage)
        float
            Swap utilisation (percentage)
        """
        return (
            psutil.cpu_percent(),
            psutil.virtual_memory().percent,
            psutil.swap_memory().percent,
            )

    def poll_gpus(self, flatten=False):
        """
        Query GPU utilisation, and sanitise results

        Returns
        -------
        list of lists of utilisation stats
            For each GPU (outer list), there is a list of utilisations
            corresponding to each query (inner list), as a string.
        """
        res = subprocess.check_output(
            ['nvidia-smi',
             '--query-gpu=' + self.gpu_query,
             '--format=csv,nounits,noheader']
            )
        lines = [i_res for i_res in res.decode().split('\n') if i_res != '']
        data = [[val.strip() if 'Not Supported' not in val else 'N/A'
                 for val in line.split(',')
                 ] for line in lines]
        if flatten:
            data = [y for row in data for y in row]
        return data

    def write_record(self):
        with smart_open(self.fname, 'a') as hf:
            stats = list(self.poll_cpu())
            if self.date_format is None:
                t = '{:.3f}'.format(time.time())
            else:
                t = time.strftime(self.date_format)
            stats.insert(0, t)
            stats += self.poll_gpus(flatten=True)
            if self.style == 'csv':
                print(','.join([str(stat) for stat in stats]), file=hf)
            elif self.style == 'tabular':
                print(self.tabular_format.format(*stats), file=hf)
            else:
                raise ValueError('Unrecognised style: {}'.format(self.style))

    def __call__(self, n_iter=None):
        if self.show_header and (self.header_count < 1
                                 or not self.header_only_once):
            self.write_header()
        n_iter = self.iter_limit if n_iter is None else n_iter
        i_iter = 0
        while True:
            t_begin = time.time()
            self.write_record()
            i_iter += 1
            if n_iter is not None and n_iter > 0 and i_iter >= n_iter:
                break
            t_sleep = self.refresh_interval + t_begin - time.time() - 0.001
            if t_sleep > 0:
                time.sleep(t_sleep)

def main(**kwargs):
    logger = Logger(**kwargs)
    logger()

if __name__ == "__main__":
    import argparse

    parser = argparse.ArgumentParser()
    parser.add_argument(
        '-l', '--loop',
        help = 'Loop every INTERVAL seconds. Default is 1 second.',
        metavar = 'INTERVAL',
        dest = 'refresh_interval',
        type = float,
        default = 1.0,
    )
    parser.add_argument(
        '-n', '--niter',
        help = 'Repeat at most MAXITER times. Default is -1, which'
               ' corresponds to forever (send interrput signal to terminate).',
        metavar = 'MAXITER',
        dest = 'iter_limit',
        type = int,
        default = -1,
    )
    parser.add_argument(
        '-H', '--no-header',
        help = 'Disable header in table output. Default is enabled.',
        dest = 'show_header',
        action = 'store_false',
    )
    parser.add_argument(
        '--header',
        help = 'Enable header in table output. Default is enabled.',
        dest = 'show_header',
        action = 'store_true',
        default = True,
    )
    parser.add_argument(
        '-c', '--csv',
        help = 'Output in csv style. Default style is csv if FILE is set.',
        dest = 'style',
        action = 'store_const',
        const = 'csv',
    )
    parser.add_argument(
        '-t', '--tabular',
        help = 'Output in ascii table style. Default style is tabular'
               ' if FILE is not set (output is sent to stdout).',
        dest = 'style',
        action = 'store_const',
        const = 'tabular',
    )
    parser.add_argument(
        '-s', '--date-seconds',
        help = 'Show time in seconds since Unix epoch (with ms precision'
               ' reported). Enabled by default.',
        dest = 'date_format',
        action = 'store_const',
        const = None,
    )
    parser.add_argument(
        '-I', '--date-iso',
        help = 'Show time in ISO format.',
        dest = 'date_format',
        action = 'store_const',
        const = '%Y-%m-%dT%H:%M:%S%z',
    )
    parser.add_argument(
        '--date-local',
        help = 'Show time in ISO-compliant, but more human friendly format,'
               ' omitting the time zone component and using a space as date/'
               'time separator.',
        dest = 'date_format',
        action = 'store_const',
        const = '%Y-%m-%d %H:%M:%S',
    )
    parser.add_argument(
        '-d', '--date-custom',
        help = 'Show time in custom format, FORMAT.',
        dest = 'date_format',
        action = 'store',
    )
    parser.add_argument(
        '--units',
        help = 'In the header, include units for each column.'
               ' Enabled by default.',
        dest = 'show_units',
        action = 'store_true',
        default = True,
    )
    parser.add_argument(
        '--no-units',
        help = 'In the header, omit units. Default is to include units.',
        dest = 'show_units',
        action = 'store_false',
        default = True,
    )
    parser.add_argument(
        'fname',
        help = 'If present, output is appended to FILE. If omitted,'
               ' output is sent to stdout.',
        metavar = 'FILE',
        default = None,
        nargs = '?',
    )
    args = parser.parse_args()
    main(**vars(args))

How to beautify the web page or HTML using bootstrap?

https://getbootstrap.com/docs/3.3/getting-started/

Thursday 21 April 2022

how to use vim to make a proper python indentation in mac OS?

alias python="python3"

brew install vim

vi ~/.vimrc

syntax on
set showmatch
set ts=4
set sts=4
set sw=4
set autoindent
set smartindent
set smarttab
set expandtab
set number

(close the terminal and open again)

vim temp.py

Wednesday 13 April 2022

How to install the latest cmake version?

reference: https://askubuntu.com/questions/355565/how-do-i-install-the-latest-version-of-cmake-from-the-command-line

Kitware now has an APT repository that currently supports Ubuntu 16.04, 18.04 and 20.04

Install Instructions:

Remove old version of cmake
```
sudo apt purge --auto-remove cmake
```

Obtain a copy of the signing key

wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | gpg --dearmor - | sudo tee /etc/apt/trusted.gpg.d/kitware.gpg >/dev/null

Add the repository to your sources list

a. For Ubuntu Focal Fossa (20.04)

sudo apt-add-repository 'deb https://apt.kitware.com/ubuntu/ focal main'

b. For Ubuntu Bionic Beaver (18.04)

sudo apt-add-repository 'deb https://apt.kitware.com/ubuntu/ bionic main'

c. For Ubuntu Xenial Xerus (16.04)

sudo apt-add-repository 'deb https://apt.kitware.com/ubuntu/ xenial main'

Update and install
```
sudo apt update
sudo apt install cmake
```

Wednesday 6 April 2022

QXcbConnection: XCB error: 2 (BadValue), sequence: 499, resource id: 1920, major code: 130 (Unknown), minor code: 3

export XDG_RUNTIME_DIR=/usr/lib
export QT_LOGGING_RULES='*.debug=false;qt.qpa.*=false'

Monday 4 April 2022

ImportError: /usr/lib/x86_64-linux-gnu/libQt5Core.so.5: version `Qt_5.15' not found (required by /home/ninja/anaconda3/envs/cloning/lib/python3.7/site-packages/PyQt5/QtCore.abi3.so)

pip uninstall pyqt5

pip install pyqt5==5.12.0