transfer file between windows and ubuntu - filezilla

sometime it is time consuming to upload the data or file using mobixterm

may be you can try use filezilla and following are the settings:-
host: sftp://10.80.43.31
username: name
password: word:
port: 22

(image source: https://filezilla.net/)

sudo apt update - proxy

how to use proxy in ubuntu?

here are few commands that might help
env | grep proxy
unset no_proxy
sudo nano /etc/environment
  PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
  http_proxy="http://proxy.company.com:port/"
  https_proxy="https://proxy.company.com:port/"
  ftp_proxy="ftp://proxy.company.com:port/"

env | grep proxy
sudo apt update
sudo apt upgrade

sudo nano /etc/apt/sources.list (make sure use the default settings from another computer)

pyinstaller

In order to convert the python code to exe in windows, we can use pyinstaller.

Here are the steps:
1. Make sure you can run the python code under windows.
If you are running with cuda, make sure the device id is selected correctly, 0/1.

2. Once you can run the python code in windows, next is to install the pyinstaller.
It must be run under python > 3.3 and you can install it as following:
$ pip install pyinstaller

3. Once you have the pyinstaller, next you need to setup a main.spec.
Assume your python code is main.py.
You can create a main.spec file as following:

# -*- mode: python -*-
#filename = Main.spec
import sys 
myLibPath = './libs'
sys.path.append(myLibPath)
a = Analysis(['Main.py'],
             pathex=['/Users/.../ProgramNameDir'],
             hiddenimports=[],
             hookspath=None,
             runtime_hooks=None)
pyz = PYZ(a.pure)
exe = EXE(pyz,
Tree('img',prefix='img'), 
          a.scripts,
          a.binaries,
          a.zipfiles,
          a.datas,
          name='Main',
          debug=False,
          strip=None,
          upx=True,
          console=False , icon='mainIcon.ico')
app = BUNDLE(exe,
             name='Main.app',
　　　　　　  info_plist={ 'NSHighResolutionCapable': 'True'}, #<-- Option for High Resolution
             icon='mainIcon.ico')

4. Once you setup the main.spec, then you are all set and ready to compile it.
Run the following command to convert main.py into main.exe.
Output will be generated in two folders, build and dist.
You only need to look at the dist folder > main.exe.
 $ pyinstaller main.spec

5. During compilation, you may receive some errors.
Most probably is due to the missing packages under python.
For example, I need to run the following code to install some packages.
$ pip install setuptools --upgrade
$ pip uninstall six
$ pip install six
$ conda install --channel https://conda.anaconda.org/bpentz pyqt5

6. Once compilation completed, congratulations, you are done.
You can navigate to root > dist > main.exe.
If you running the main.py with some additional files, remember to copy into dist folder.
You may run the exe in cmd to see the errors.

p/s: I remembered the compiled exe cannot be used for commercial purpose as it uses some Microsoft dll or licensed dll, please be aware.

from device: CUDA_ERROR_OUT_OF_MEMORY

2019-03-06 10:15:43.999337: E C:\users\nwani\_bazel_nwani\swultrt5\execroot\org_tensorflow\tensorflow\stream_executor\cuda\cuda_driver.cc:936] failed to allocate 2.3K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2019-03-06 10:15:43.999337: E C:\users\nwani\_bazel_nwani\swultrt5\execroot\org_tensorflow\tensorflow\stream_executor\cuda\cuda_driver.cc:936] failed to allocate 2.3K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2019-03-06 10:15:43.999337: E C:\users\nwani\_bazel_nwani\swultrt5\execroot\org_tensorflow\tensorflow\stream_executor\cuda\cuda_driver.cc:936] failed to allocate 2.3K (2304 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2019-03-06 10:15:44.000337: E C:\users\nwani\_bazel_nwani\swultrt5\execroot\org_tensorflow\tensorflow\stream_executor\cuda\cuda_driver.cc:936] failed to allocate 2.3K (2304 bytes) from

Check the nvidia driver version, cuda version, cudnn version, python version, tensorflow version

Under windows, you can change the tensorflow version by following in cmd:
$ activate tensorflow
$ pip install tensorflow==1.4.0

Softmax Intuition

Given a classification model, the output will produce the confidence score for each class instead of the original value of forward propagation.

For example, model predict a = -10, b = 10, c = 1.
Softmax will convert the output to confidence score as following.
a1 = e^a / (e^a + e^b + e^c)
b1 = e^b / (e^a + e^b + e^c)
c1 = e^c / (e^a + e^b + e^c)

As a result, the model will produce the confidence score of class a, b, c as a1, b1, c1.

[image source: https://cdn-images-1.medium.com/max/2000/1*eqQuFgXPUP5L6J_vVH19wg.png]

Protobuf Error @@

This error often happen when building opencv or caffe.
I found it is quite annoying as everytime I have to spend a lot of time to debug.
I think I understand the issue now.

First, check the protoc version you are using:-
$ protoc --version
libprotoc 2.6.1
$ dpkg -l | grep protobuf

Second, check which protoc you are using, remove those not from /usr/bin/:-

$ which protoc
/home/user/.conda/envs/caffe2/bin/protoc
$ conda uninstall libprotobuf


Third, remove and install again the protoc and protobuf:-

$ sudo add-apt-repository ppa:maarten-fonville/protobuf 
$ sudo apt-get update
$ sudo apt-get remove libprotobuf-dev libprotoc-dev protobuf-compiler
$ sudo apt-get install libprotobuf-dev libprotoc-dev protobuf-compiler 
Finally rerun the make for opencv and caffe.
One thing to take note for opencv as there are some conflict when enabling the dnn module.
If there is an error of protobuf when building opencv, try add on the following to cmake:-
-DBUILD_PROTOBUF=OFF
-DPROTOBUF_UPDATE_FILES=ON
-DBUILD_OPENCV_DNN=OFF

platform: ubuntu16.04, python2.7, anaconda3

Intuition

Most of the time, I will write down all the coding experience or error in a small book.
However, I found that it is difficult to organize and to track.
Sometime, I will write some rubbish on my notebook, it makes the things worse.
I used to write the blog long time ago, but I stop for quite a long time.
I decided to get back to blog and write down my experience.
You know, time flies, age is getting older, memory is getting lousy.
So, let's get started!