How to copy a file to docker container?

1. to check the container id

# docker ps

2. to copy a file from host to container

# sudo docker cp ./test.mp4 73411ab49381:/workspace/input

How to configure a Makefile with Opencv?

 First compile the opencv using the previous post

Second, modify the Makefile as following:-

all:demo

CFLAGS = `pkg-config --cflags opencv4`
LIBS = `pkg-config --libs opencv4`

SRC:=main.cpp  file2.cpp  file3.cpp  file4.cpp
demo:$(SRC)
    g++ $(CFLAGS) -std=c++11 -Wall -O2  $^  -c -Iffmpeg/include/  -I rtspperf/inc
    g++ $(LIBS) -g *.o -o main -lopencv_core -lopencv_imgproc -lopencv_imgcodecs -ldl -lz -lm  -llzma
    @rm *.o -f

clean:
    rm -f demo

PCIe versus NVLink

 For normal PC, we are using PCIe 3.0 for connecting to 2080Ti.

For training, it is better using NVLink and the data throughput is much faster than PCIe.

However, for deployment or model inference, PCIe is more than enough.

How to do empty trash in ubuntu using terminal?

 sudo apt install trash-cli

 trash-empty

Windows - MobileXterm / Linux - SnowFlake

https://mobaxterm.mobatek.net/

 https://github.com/subhra74/snowflake

 

How to debug the segmentation fault in cpp?

ref: https://stackoverflow.com/questions/8305866/how-do-i-analyze-a-programs-core-dump-file-with-gdb-when-it-has-command-line-pa

 

If you want to pass parameters to the executable to be debugged in GDB, use --args.

For example:

$ gdb --args ./crash -p param1 -o param2
GNU gdb (GDB) 7.1-ubuntu
...
(gdb) r
Starting program: /home/@@@@/crash -p param1 -o param2

Program received signal SIGSEGV, Segmentation fault.
__strlen_ia32 () at ../sysdeps/i386/i686/multiarch/../../i586/strlen.S:99
99    ../sysdeps/i386/i686/multiarch/../../i586/strlen.S: No such file or directory.
    in ../sysdeps/i386/i686/multiarch/../../i586/strlen.S
(gdb)

cannot find -lopencv, no package 'opencv' found? opencv.pc? PKG_CONFIG_PATH?

check this location /home/ninja/workspace/opencv-3.4.4/distribute/lib/pkgconfig

export PKG_CONFIG_PATH=/home/ninja/workspace/opencv-3.4.4/distribute/lib/pkgconfig

How to store python data using hickle? hickle = pickle + hdf5?

import os
import hickle as hkl
import numpy as np

# create a numpy array of data
array_obj = np.ones(32768, dtype='float32')

# dump to file
hkl.dump(array_obj, 'test.hkl', mode='w')

# dump data, with compression
hkl.dump(array_obj, 'test_gzip.hkl', mode='w', compression='gzip')

# compare filesizes
print('uncompressed: {} bytes'.format(os.path.getsize('test.hkl')))
print('compressed: {} bytes'.format(os.path.getsize('test_gzip.hkl')))

# load data
array_hkl = hkl.load('test_gzip.hkl')

# check the two are the same file
assert array_hkl.dtype == array_obj.dtype
assert np.all((array_hkl, array_obj))


# ===
# hdf5 compression options
# datasets are stored as B-trees, data are split into chunks
# fletcher32=True computes a checksum
# scaleoffset=0, move the data around to improve compression ratios
# hkl.dump(array_obj, 'test_lzf.hkl', mode='w', compression='lzf', scaleoffset=0, \
#     chunks=True, shuffle=True, fletcher32=True)

How to connect shared folder in Windows? How to Map Network Drives in Windows 10?

p/s: copied from https://ccit.college.columbia.edu/knowledgebase/article/how-map-network-drives-windows-10

Press Windows key to open Search. Type This PC into the Search.

Click Computer at the top. Select Map network drive.

In the Map Network Drive window choose which Drive Letter you wish to assign to the Network Drive you're currently mapping. In the Folder field, enter the full network path you wish to map (this path should be provided to you by your CC sponsor or CCIT personnel). As shown in the screen shot below, be sure to check the boxes for Reconnect at sign-in and Connect using different credentials. When all fields are filled out, click Finish

​For Columbia College Staff, the most commonly used shared drives have the following addresses:

You'll be asked to enter a set of credentials to access the Network share you're trying to map. At this time, enter your COLLEGE login credentials. Be sure to enter the full domain path before your UNI, as shown in the screen shot below. Check the box Remember my credentials, then click OK.

When completed the Network share should be mapped for you and will be accessible from the This PC window.

---

How to check ubuntu motherboard?

 sudo dmidecode | grep Product

How to save opencv Mat data and read in python?

#include <opencv2/hdf.hpp>
using namespace std;

static void write_multiple_channels(cv::Mat data)
{
    std::string filename = "data.h5";
    std::string parent_name = "/data";
    std::string dataset_name = parent_name + "/two_channels";
    // prepare data
    for (size_t i = 0; i < data.total()*data.channels(); i++)
        ((float*) data.data)[i] = (float)i;
    cv::Ptr<cv::hdf::HDF5> h5io = cv::hdf::open(filename);
    // first we need to create the parent group
    if (!h5io->hlexists(parent_name)) h5io->grcreate(parent_name);
    // create the dataset if it not exists
    if (!h5io->hlexists(dataset_name)) h5io->dscreate(data.rows, data.cols, data.type(), dataset_name);
    // the following is the same with the above function write_root_group_single_channel()
    h5io->dswrite(data, dataset_name);
    cv::Mat expected;
    h5io->dsread(expected, dataset_name);
    double diff = norm(data - expected);
    CV_Assert(abs(diff) < 1e-10);
    h5io->close();
}

===

import h5py    
import numpy as np    
h5data = h5py.File("data.h5",'r')  
temp = h5data.get('/data/two_channels')
temp = np.array(temp)

import pdb; pdb.set_trace()

 

How to create a bash script with conda activate?

 source ~/anaconda3/etc/profile.d/conda.sh
conda activate python2
python parse_log.py ../train.log .
python plot_loss.py

How to view different networks structure?

Netron is a viewer for neural network, deep learning and machine learning models.

Netron supports ONNX (.onnx, .pb, .pbtxt), Keras (.h5, .keras), Core ML (.mlmodel), Caffe (.caffemodel, .prototxt), Caffe2 (predict_net.pb), Darknet (.cfg), MXNet (.model, -symbol.json), Barracuda (.nn), ncnn (.param), Tengine (.tmfile), TNN (.tnnproto), UFF (.uff) and TensorFlow Lite (.tflite).

Netron has experimental support for TorchScript (.pt, .pth), PyTorch (.pt, .pth), Torch (.t7), Arm NN (.armnn), BigDL (.bigdl, .model), Chainer (.npz, .h5), CNTK (.model, .cntk), Deeplearning4j (.zip), MediaPipe (.pbtxt), ML.NET (.zip), MNN (.mnn), PaddlePaddle (.zip, __model__), OpenVINO (.xml), scikit-learn (.pkl), TensorFlow.js (model.json, .pb) and TensorFlow (.pb, .meta, .pbtxt, .ckpt, .index).

macOS: Download the .dmg file or run brew cask install netron

Linux: Download the .AppImage file or run snap install netron

Windows: Download the .exe installer or run winget install netron

Browser: Start the browser version.

Python Server: Run pip install netron and netron [FILE] or netron.start('[FILE]').

 

Sample model files to download or open using the browser version:

• ONNX: squeezenet [open]
• CoreML: exermote [open]
• Darknet: yolo [open]
• Keras: mobilenet [open]
• MXNet: inception_v3 [open]
• TensorFlow: chessbot [open]
• TensorFlow Lite: hair_segmentation [open]
• TorchScript: traced_online_pred_layer [open]
• Caffe: mobilenet_v2 [open]

 

reference: https://github.com/lutzroeder/netron

 

How to do os recovery for Jetson board?

Preparing a Jetson Developer Kit for Use

This section explains how to prepare a Jetson developer kit for use by flashing it with the appropriate software.

Assumptions

• You have a Jetson developer kit.

For the Jetson Nano Developer Kit, you can skip this guide and simply download and use the supported microSD card image. Alternatively, you can follow these instructions to flash the QSPI-NOR, or flash the QSPI-NOR and a microSD card inserted on the Jetson Nano module.

• Your Jetson developer kit is powered off, and is connected as follows. (Note that your Jetson developer kit may not come with the devices and cables listed below.)

• A USB cable connects the correct USB port on your Jetson developer kit to your Linux host for flashing.

For the NVIDIA Jetson AGX Xavier Developer Kit, use the USB-C port next to the power button. For the other Jetson developer kits, use the micro USB port.

• Any required USB peripherals such as keyboard and mouse are connected to the Jetson developer kit, possibly through a USB hub.

• A wired Ethernet connection is available for installing optional software on the Jetson developer kit after L4T is installed and running.

• Either a display device or a serial console is connected to the Jetson developer kit.

• The qemu-user-static package has been installed on the Linux host:

sudo apt-get install qemu-user-static

The installation process needs this package to install certain NVIDIA software components onto the Jetson developer kit.

Environment Variables

The directions that follow assume that:

• ${L4T_RELEASE_PACKAGE} contains the L4T release package’s name.

• For Jetson Nano and Jetson TX1 modules: Jetson-210_Linux_r32.4.2_aarch64.tbz2

• For Jetson Xavier NX, Jetson AGX Xavier series, and Jetson TX2 series modules: Jetson_Linux_L4T r32.4.2_aarch64.tbz2

• ${SAMPLE_FS_PACKAGE} contains the filename of the sample filesystem package:

Tegra_Linux_Sample-Root-Filesystem_L4TRELEASEVERSION.tbz2

• ${BOARD} contains the name of a configuration supported for your specific Jetson module and carrier board.

Some examples are:

• jetson-nano-devkit

• jetson-tx2-devkit

• jetson-xavier-nx-devkit

Recent and future configuration files’ names follow this format:

<board_part_number>+<module_part_number>[-<modifier>].conf

• <board_part_number> is the P‑number (the full part number) for the NVIDIA carrier board that the .conf file supports.

• <module_part_number> is the P‑number for the Jetson module that the .conf file supports.

• <modifier> is an optional string that indicates which of several configurations the .conf file applies; for example, on a module that may have either eMMC or microSD card memory, which one it uses to flash the operating software.

Older configuration files’ name generally follow this format:

<config_name>[-<modifier>].conf

Where <config_name> is a name that represents the carrier board and module that the .conf file supports.

To flash Jetson Developer Kit operating software

1. Download the latest L4T release package and sample file system for your Jetson developer kit from:

https://developer.nvidia.com/linux-tegra

2. Enter these commands to untar the files and assemble the rootfs:

$ tar xf ${L4T_RELEASE_PACKAGE} or tar xf Tegra186_Linux_R32.4.3_aarch64.tbz2

$ cd Linux_for_Tegra/rootfs/

$ sudo tar xpf ../../${SAMPLE_FS_PACKAGE} or sudo tar xpf Tegra_Linux_Sample-Root-Filesystem_R32.4.3_aarch64.tbz2

$ cd ..

$ sudo ./apply_binaries.sh

3. Ensure that your Jetson developer kit is configured and connected to your Linux host as described in Assumptions.

4. Put your Jetson developer kit into Force Recovery Mode (RCM).

 

For other Jetson developer kits that have hardware buttons:

1) Ensure that the developer kit is powered off.

2) Press and hold down the Force Recovery button.

3) Press, then release the Power button.

4) Release the Recovery button.

5) Continue the software installation.

You can confirm that your developer kit is in Force Recovery mode by following the procedure in To determine whether the developer kit is in Force Recovery mode.

5. Enter this command on your Linux host to flash (install) the L4T release onto the Jetson developer kit:

sudo ./flash.sh ${BOARD} mmcblk0p1

sudo ./flash.sh jetson-xavier-nx-devkit-emmc mmcblk0p1

The value of the environment variable $(BOARD) determines the configuration that the flashing script applies. The table at the end of this section describes the configurations available for each Jetson module with its respective Jetson developer kit carrier board.

There is a configuration file that corresponds to each value of $(BOARD). Its name is the value of $(BOARD) with the filename $(BOARD).conf. For example, the configuration file that corresponds to jetson-nano-devkit is jetson-nano-devkit.conf.

Flashing L4T takes about 10 minutes, or more with a slow host computer.

 

Boot into Jetson OS

sudo apt update

sudo apt upgrade

sudo apt install nvidia-jetpack

 

 

Check Cuda, Tensorrt, CUDNN Status

sudo apt install python-pip

sudo pip install jetson-stats

jtop

Look at the page info (6)

 

All Done!

references:-

https://developer.nvidia.com/embedded/linux-tegra
https://developer.nvidia.com/embedded/L4T/r32_Release_v4.3/t186ref_release_aarch64/Tegra186_Linux_R32.4.3_aarch64.tbz2
https://developer.nvidia.com/embedded/L4T/r32_Release_v4.3/t186ref_release_aarch64/Tegra_Linux_Sample-Root-Filesystem_R32.4.3_aarch64.tbz2

recovery instructions:-
https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fquick_start.html%23wwpID0E0BE0HA

How to boot from SSD after sdcard flash

https://www.seeedstudio.com/blog/2020/06/22/boot-jetson-xavier-from-m-2-ssd/ 

 

Step 1. Format the SSD you have installed

Install your and SSD and boot, then open the menu and search disk.

Step 2. Copy the rootfs from SD card to SSD

Firstly, clone the project.

git clone https://github.com/jetsonhacks/rootOnNVMe.git
cd rootOnNVMe

Then copy rootfs file to your SSD

./copy-rootfs-ssd.sh

Step 3. Enable booting from SSD

Run ./setup-service.sh and reboot to make the service come into effect.