the reason is due to the missing dependencies of boost date_time library in cmake configuration
Goto root > cmake > Dependencies.cmake
search a keyword: filesystem (original source)
find_package(Boost 1.61 COMPONENTS "python${PYTHON_VERSION_MAJOR}" system thread filesystem regex)
adding a keyword named as date_time into all corresponding search result (modified source):-
find_package(Boost 1.61 COMPONENTS "python${PYTHON_VERSION_MAJOR}" date_time system thread filesystem regex)
Thursday, 9 May 2019
Trace or track Python statement execution (trace)
Function in the 'trace' module in Python library generates trace of
program execution, and annotated statement coverage. It also has
functions to list functions called during run by generating caller
relationships.
Following two Python scripts are used as an example to demonstrate features of trace module.
Following two Python scripts are used as an example to demonstrate features of trace module.
#myfunctions.py import math def area(x): a = math.pi*math.pow(x,2) return a def factorial(x): if x==1: return 1 else: return x*factorial(x-1)
#mymain.py import myfunctions def main(): x = 5 print ('area=',myfunctions.area(x)) print ('factorial=',myfunctions.factorial(x)) if __name__=='__main__': main()
The 'trace' module has
a command line interface. All the functions in the module can be called
using command line switches. The most important option is --trace which displays program lines as they are executed. In following example another option --ignore-dir is used. It ignores specified directories while generating the trace.E:\python37>python -m trace --ignore-dir=../lib --trace mymain.py
Output
mymain.py(2): def main():
mymain.py(7): if __name__=='__main__':
mymain.py(8): main()
--- modulename: mymain, funcname: main
mymain.py(3): x=5
mymain.py(4): print ('area=',myfunctions.area(x))
--- modulename: myfunctions, funcname: area
myfunctions.py(3): a=math.pi*math.pow(x,2)
myfunctions.py(4): return a
area= 78.53981633974483
mymain.py(5): print ('factorial=',myfunctions.factorial(x))
--- modulename: myfunctions, funcname: factorial
myfunctions.py(6): if x==1:
myfunctions.py(9): return x*factorial(x-1)
--- modulename: myfunctions, funcname: factorial
myfunctions.py(6): if x==1:
myfunctions.py(9): return x*factorial(x-1)
--- modulename: myfunctions, funcname: factorial
myfunctions.py(6): if x==1:
myfunctions.py(9): return x*factorial(x-1)
--- modulename: myfunctions, funcname: factorial
myfunctions.py(6): if x==1:
myfunctions.py(9): return x*factorial(x-1)
--- modulename: myfunctions, funcname: factorial
myfunctions.py(6): if x==1:
myfunctions.py(7): return 1
factorial= 120
The --count option generates a file for each module in use with, cover extension.
E:\python37>python -m trace --count mymain.py area= 78.53981633974483 factorial = 120myfunctions.cover
1: import math 1: def area(x): 1: a = math.pi*math.pow(x,2) 1: return a 1: def factorial(x): 5: if x==1: 1: return 1 else: 4: return x*factorial(x-1)mymain.cover
1: import myfunctions
1: def main():
1: x = 5
1: print ('area=',myfunctions.area(x))
1: print ('factorial=',myfunctions.factorial(x))
1: if __name__=='__main__':
1: main()
--summary option displays a brief summary if –count option is also used.
E:\python37>python -m trace --count --summary mymain.py area = 78.53981633974483 factorial = 120 lines cov% module (path) 8 100% myfunctions (E:\python37\myfunctions.py) 7 100% mymain (mymain.py)
The --file option specifies name of file in which accumulates count over several tracing runs.
E:\python37>python -m trace --count --file report.txt mymain.py area = 78.53981633974483 factorial = 120 Skipping counts file 'report.txt': [Errno 2] No such file or directory: 'report.txt' E:\python37>python -m trace --count --file report.txt mymain.py area= 78.53981633974483 factorial= 120
--listfuncs option displays functions called during execution of program.E:\python37>python -m trace --listfunc mymain.py | findstr -v importlib area= 78.53981633974483 factorial= 120 functions called: filename: E:\python37\lib\encodings\cp1252.py, modulename: cp1252, funcname: IncrementalEncoder.encode filename: E:\python37\myfunctions.py, modulename: myfunctions, funcname: <module> filename: E:\python37\myfunctions.py, modulename: myfunctions, funcname: area filename: E:\python37\myfunctions.py, modulename: myfunctions, funcname: factorial filename: mymain.py, modulename: mymain, funcname: <module> filename: mymain.py, modulename: mymain, funcname: main
--trackcalls option is used along with –list funcs option. It generates calling relationships.E:\python37>python -m trace --listfunc --trackcalls mymain.py | findstr -v importlib area= 78.53981633974483 factorial= 120 calling relationships: --> E:\python37\myfunctions.py *** E:\python37\lib\trace.py *** --> mymain.py trace.Trace.runctx -> mymain.<module> *** E:\python37\myfunctions.py *** myfunctions.factorial -> myfunctions.factorial *** mymain.py *** mymain.<module> -> mymain.main --> E:\python37\lib\encodings\cp1252.py mymain.main -> cp1252.IncrementalEncoder.encode --> E:\python37\myfunctions.py mymain.main -> myfunctions.area mymain.main -> myfunctions.factorial
(ref: https://www.tutorialspoint.com/trace-or-track-python-statement-execution-trace)
Tuesday, 7 May 2019
ubuntu programming checklist
Ubuntu is well known for deep learning programming or development.
However, this os can be easily corrupted by few sudo commands.
Therefore, here is a checklist for myself before and after some works.
1. use mobaxterm and tmux for remote programming
2. use github for source code management
3. write down steps used for a success or failure trial
4. always zip the images before transferring, especially something very small like 24x24
5. always check the path before running a time consuming program
However, this os can be easily corrupted by few sudo commands.
Therefore, here is a checklist for myself before and after some works.
1. use mobaxterm and tmux for remote programming
2. use github for source code management
3. write down steps used for a success or failure trial
4. always zip the images before transferring, especially something very small like 24x24
5. always check the path before running a time consuming program
Monday, 6 May 2019
how to build caffe in ubuntu16, cuda10.0, cudnn7.6, python2.7
0. install the compatible nvidia driver, cudnn and nccl for ubuntu16. They are
cuda10.0 (cuda_10.0.130_410.48_linux.run) and cudnn7.6.0 (cudnn-10.0-linux-x64-v7.6.0.64.tgz) and nccl (nccl_2.4.7-1+cuda10.0_x86_64.txz).
0.1 install general dependencies:
sudo apt-get remove x264 libx264-dev
sudo apt update
sudo apt upgrade
sudo apt install python2.7 python-pip
sudo apt install build-essential checkinstall cmake pkg-config yasm git gfortran libjpeg8-dev libjasper-dev libpng12-dev libtiff5-dev libavcodec-dev libavformat-dev libswscale-dev libdc1394-22-dev libxine2-dev libv4l-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libgtk2.0-dev libtbb-dev libatlas-base-dev libfaac-dev libmp3lame-dev libtheora-dev libvorbis-dev libxvidcore-dev libopencore-amrnb-dev libopencore-amrwb-dev x264 v4l-utils libgoogle-glog-dev libgflags-dev libgphoto2-dev libeigen3-dev libhdf5-dev doxygen
sudo apt-get install ffmpeg libgtk-3-dev python-numpy python3-numpy libdc1394-22 libdc1394-22-dev libjpeg-dev libpng12-dev libtiff5-dev libjasper-dev libavcodec-dev libavformat-dev libswscale-dev libxine2-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libv4l-dev libtbb-dev qtbase5-dev libfaac-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-dev x264 v4l-utils unzip liblmdb-dev libleveldb-dev libsnappy-dev libboost-dev
sudo apt install qt5-default
sudo apt install libboost-all-dev
1. install the correct protobuf packages (version 3.6.1) as given:-
3. create a conda virtual env named as caffe2 with python2.7, install:
$ pip install scikit-image
$ pip install protobuf (after completed step 2)
4. download caffe and build it with 'cmake ..' and 'make -j$(nproc)' and 'sudo make install'
$ cd caffe/
$ cd build/
$ sudo rm -r ./*
$ cmake .. -DCMAKE_INSTALL_PREFIX:PATH=/home/ninja/.conda/envs/caffe
$ make -j32
$ sudo make install
$ make runtest
$ make pycaffe
4.1 if sudo make install failed due to undefined reference to `ZSTD_initCStream, run:
$ conda list
$ conda install libtiff (it should install zstd)
or you can just modify the library path by pointing to system libraries only
$ gedit ~/.bashrc
>> # insert into last line
>> export $LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu
$ source ~/.bashrc
$ conda activate caffe2(compile again, make -j32)
4.2 C++ exception with description "locale::facet::_S_create_c_locale name not valid" thrown in SetUp().
$ export LC_ALL="C"
4.3 If you received the following error:-
CXX/LD -o .build_release/tools/convert_imageset.bin .build_release/lib/libcaffe.so: undefined reference to `google::protobuf::io::CodedOutputStream::WriteVarint64ToArray(unsigned long long, unsigned char*)' .build_release/lib/libcaffe.so: undefined reference to `google::protobuf::internal::WireFormatLite::WriteInt64(int, long long, google::protobuf::io::CodedOutputStream*)' .build_release/lib/libcaffe.so: undefined reference to `google::protobuf::UnknownFieldSet::AddVarint(int, unsigned long long)' .build_release/lib/libcaffe.so: undefined reference to `google::protobuf::io::CodedOutputStream::VarintSize64(unsigned long long)' .build_release/lib/libcaffe.so: undefined reference to `google::protobuf::io::CodedOutputStream::WriteVarint64(unsigned long long)' collect2: error: ld returned 1 exit status make: *** [.build_release/tools/convert_imageset.bin] Error 1
9. how to force cmake to use one particular cuda architecture?
-DCUDA_ARCH_NAME=Manual -DCUDA_ARCH_BIN="75" -DCUDA_ARCH_PTX="75"
cuda10.0 (cuda_10.0.130_410.48_linux.run) and cudnn7.6.0 (cudnn-10.0-linux-x64-v7.6.0.64.tgz) and nccl (nccl_2.4.7-1+cuda10.0_x86_64.txz).
0.1 install general dependencies:
sudo apt-get remove x264 libx264-dev
sudo apt update
sudo apt upgrade
sudo apt install python2.7 python-pip
sudo apt install build-essential checkinstall cmake pkg-config yasm git gfortran libjpeg8-dev libjasper-dev libpng12-dev libtiff5-dev libavcodec-dev libavformat-dev libswscale-dev libdc1394-22-dev libxine2-dev libv4l-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libgtk2.0-dev libtbb-dev libatlas-base-dev libfaac-dev libmp3lame-dev libtheora-dev libvorbis-dev libxvidcore-dev libopencore-amrnb-dev libopencore-amrwb-dev x264 v4l-utils libgoogle-glog-dev libgflags-dev libgphoto2-dev libeigen3-dev libhdf5-dev doxygen
sudo apt-get install ffmpeg libgtk-3-dev python-numpy python3-numpy libdc1394-22 libdc1394-22-dev libjpeg-dev libpng12-dev libtiff5-dev libjasper-dev libavcodec-dev libavformat-dev libswscale-dev libxine2-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libv4l-dev libtbb-dev qtbase5-dev libfaac-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-dev x264 v4l-utils unzip liblmdb-dev libleveldb-dev libsnappy-dev libboost-dev
sudo apt install qt5-default
sudo apt install libboost-all-dev
1. install the correct protobuf packages (version 3.6.1) as given:-
# Make sure you grab the latest version
wget https://github.com/google/protobuf/releases/download/v3.6.1/protoc-3.6.1-linux-x86_64.zip
# Unzip
unzip protoc-3.6.1-linux-x86_64.zip -d protoc3
# Move protoc to /usr/local/bin/
sudo cp -r protoc3/bin/* /usr/local/bin/
# Move protoc3/include to /usr/local/include/
sudo cp -r protoc3/include/* /usr/local/include/
# Optional: change owner
sudo chown $USER /usr/local/bin/protoc
sudo chown -R $USER /usr/local/include/google
sudo ldconfig
1.1 (optional, ignore) install the correct protobuf packages (version 3.1.0) as given in https://ninja5code.blogspot.com/2019/02/protobuf-error.html
$ sudo add-apt-repository ppa:maarten-fonville/protobuf
$ sudo apt-get update
$ sudo apt-get remove libprotobuf-dev libprotoc-dev protobuf-compiler
$ sudo apt-get install libprotobuf-dev libprotoc-dev protobuf-compiler
$ protoc --version
$ dpkg -l | grep protobuf
2. download zip files of opencv 3.4.4 and opencv-contrib 3.4.4 from https://github.com/opencv/opencv/releases/tag/3.4.4, https://github.com/opencv/opencv_contrib/releases/tag/3.4.4. You should unzip it into the folder as /home/ninja/opencv and /home/ninja/opencv-contrib-3.4.4, respectively. Then, build it with 'cmake ..' and 'make -j$(nproc)' and 'sudo make install'
$ cd opencv
$ cd build/
$ rm -r ./*
$ cmake ..
$ make -j32
$ sudo make install
2.1 if there is any wrong reference to xxx.lib, you can list down all the dependencies using:
$ ldd /usr/local/lib/xxx.lib
2.2 Or you can check the library path to see the additional path within it:-
$ echo $LD_LIBRARY_PATH
Then, manually set it using:
$ export LD_LIBRARY_PATH = xxx:xxx
2.3 If cmake .. failed to build it, check the log printed by cmake, then install the dependencies if necessary, for example, failed to find libssl:-
$ sudo apt search libssl
$ sudo apt install libssl-dev
2.4 If make -j32 failed to build, you may try the following cmake:-
sudo cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/home/ninja/workspace/opencv-3.4.4/distribute \
-D INSTALL_C_EXAMPLES=ON \
-D OPENCV_GENERATE_PKGCONFIG=ON \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D WITH_TBB=ON \
-D WITH_V4L=ON \
-D OPENCV_PYTHON3_INSTALL_PATH=/home/ninja/workspace/opencv-3.4.4/distribute/python3 \
-D WITH_QT=ON \
-D WITH_OPENGL=ON \
-D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-3.4.4/modules \
-D BUILD_NEW_PYTHON_SUPPORT=ON \
-D BUILD_opencv_python3=ON \
-D HAVE_opencv_python3=ON \
-D PYTHON3_EXECUTABLE=/home/ninja/anaconda3/bin/python \
-D PYTHON3_DEFAULT_EXECUTABLE=/home/ninja/anaconda3/bin/python \
-D PYTHON3_INCLUDE_PATH=/home/ninja/anaconda3/include/python3.7m \
-D PYTHON3_NUMPY_INCLUDE_DIRS=/home/ninja/anaconda3/lib/python3.7/site-packages/numpy/core/include \
-D PYTHON3_PACKAGES_PATH=/home/ninja/anaconda3/lib/python3.7/site-packages \
-D PYTHON3_LIBRARIES=/home/ninja/anaconda3/lib/libpython3.7m.so \
-D PYTHON3_LIBRARIES_PATH=/home/ninja/anaconda3/lib \
-D BUILD_EXAMPLES=ON ..
(make sure you only have single version of library in LD_LIBRARY_PATH, for example, you cannot have protobuf 3.8.0 in /usr/lib/x86_64-linux-gnu/ and protobuf 3.1.0 in anaconda. You need to uninstall both and install protobuf 3.1.0 under /usr/lib/x86_64-linux-gnu only)
2.5 If you want to build with CUDA 8.0 and CUDNN 6.0 support:-
sudo cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/home/ninja/workspace/opencv-3.4.4/distribute \
-D INSTALL_C_EXAMPLES=ON \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D WITH_TBB=ON \
-D WITH_V4L=ON \
-D WITH_CUDA=ON \
-D WITH_CUBLAS=ON \
-D CUDA_FAST_MATH=ON \
-D CUDA_GENERATION=Fermi \
-D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-8.0 \
-D OPENCV_PYTHON3_INSTALL_PATH=/home/ninja/workspace/opencv-3.4.4/distribute/python3 \
-D WITH_QT=ON \
-D WITH_OPENGL=ON \
-D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-3.4.4/modules \
-D BUILD_NEW_PYTHON_SUPPORT=ON \
-D BUILD_opencv_python3=ON \
-D HAVE_opencv_python3=ON \
-D PYTHON3_EXECUTABLE=/home/ninja/anaconda3/bin/python \
-D PYTHON3_DEFAULT_EXECUTABLE=/home/ninja/anaconda3/bin/python \
-D PYTHON3_INCLUDE_PATH=/home/ninja/anaconda3/include/python3.7m \
-D PYTHON3_NUMPY_INCLUDE_DIRS=/home/ninja/anaconda3/lib/python3.7/site-packages/numpy/core/include \
-D PYTHON3_PACKAGES_PATH=/home/ninja/anaconda3/lib/python3.7/site-packages \
-D PYTHON3_LIBRARIES=/home/ninja/anaconda3/lib/libpython3.7m.so \
-D PYTHON3_LIBRARIES_PATH=/home/ninja/anaconda3/lib \
-D BUILD_EXAMPLES=ON ..
2.6 If you want to build with CUDA 10.0 and CUDNN 7.6 support:-
sudo cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/home/ninja/workspace/opencv-3.4.4/distribute \
-D INSTALL_C_EXAMPLES=ON \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D WITH_TBB=ON \
-D WITH_V4L=ON \
-D WITH_CUDA=ON \
-D WITH_CUBLAS=ON \
-D BUILD_opencv_cudacodec=OFF \
-D CUDA_FAST_MATH=ON \
-D CUDA_GENERATION=Kepler \
-D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-10.0 \
-D OPENCV_PYTHON3_INSTALL_PATH=/home/ninja/workspace/opencv-3.4.4/distribute/python3 \
-D WITH_QT=ON \
-D WITH_OPENGL=ON \
-D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-3.4.4/modules \
-D BUILD_NEW_PYTHON_SUPPORT=ON \
-D BUILD_opencv_python3=ON \
-D HAVE_opencv_python3=ON \
-D PYTHON3_EXECUTABLE=/home/ninja/anaconda3/bin/python \
-D PYTHON3_DEFAULT_EXECUTABLE=/home/ninja/anaconda3/bin/python \
-D PYTHON3_INCLUDE_PATH=/home/ninja/anaconda3/include/python3.7m \
-D PYTHON3_NUMPY_INCLUDE_DIRS=/home/ninja/anaconda3/lib/python3.7/site-packages/numpy/core/include \
-D PYTHON3_PACKAGES_PATH=/home/ninja/anaconda3/lib/python3.7/site-packages \
-D PYTHON3_LIBRARIES=/home/ninja/anaconda3/lib/libpython3.7m.so \
-D PYTHON3_LIBRARIES_PATH=/home/ninja/anaconda3/lib \
-D BUILD_EXAMPLES=ON ..
Build with Opencv-4.4.0, Opencv_Contrib-4.4.0, Ubuntu18, Cuda10-2
{
"cmake.configureArgs": [
"-DCMAKE_BUILD_TYPE=RELEASE",
"-DCMAKE_INSTALL_PREFIX=/home/ninja/workspace/opencv-4.4.0/distribute",
"-DINSTALL_C_EXAMPLES=ON",
"-DOPENCV_GENERATE_PKGCONFIG=ON",
"-DINSTALL_PYTHON_EXAMPLES=ON",
"-DBUILD_SHARED_LIBS=ON"
"-DWITH_TBB=ON",
"-DWITH_V4L=ON",
"-DBUILD_opencv_world=OFF",
"-DOPENCV_PYTHON3_INSTALL_PATH=/home/ninja/workspace/opencv-4.4.0/distribute/python3",
"-DWITH_QT=ON",
"-DWITH_OPENGL=ON",
"-DWITH_CUDA=ON",
"-DWITH_CUBLAS=ON",
"-DBUILD_opencv_cudacodec=ON",
"-DCUDA_FAST_MATH=ON",
"-DCUDA_GENERATION=Kepler",
"-DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-10.2",
"-DOPENCV_EXTRA_MODULES_PATH=/home/ninja/workspace/opencv_contrib-4.4.0/modules",
"-DBUILD_NEW_PYTHON_SUPPORT=ON",
"-DBUILD_opencv_python3=ON",
"-DHAVE_opencv_python3=ON",
"-DPYTHON3_EXECUTABLE=/home/ninja/anaconda3/bin/python",
"-DPYTHON3_DEFAULT_EXECUTABLE=/home/ninja/anaconda3/bin/python",
"-DPYTHON3_INCLUDE_PATH=/home/ninja/anaconda3/include/python3.8",
"-DPYTHON3_NUMPY_INCLUDE_DIRS=/home/ninja/anaconda3/lib/python3.8/site-packages/numpy/core/include",
"-DPYTHON3_PACKAGES_PATH=/home/ninja/anaconda3/lib/python3.8/site-packages",
"-DPYTHON3_LIBRARIES=/home/ninja/anaconda3/lib/libpython3.8.so",
"-DPYTHON3_LIBRARIES_PATH=/home/ninja/anaconda3/lib",
"-DBUILD_EXAMPLES=ON"
]
}
2.7. if you have this error: The imported target "Qt5::Gui" references the file, "/usr/lib/x86_64-linux-gnu/libEGL.so"
sudo rm /usr/lib/x86_64-linux-gnu/libEGL.so; sudo ln /usr/lib/x86_64-linux-gnu/libEGL.so.1 /usr/lib/x86_64-linux-gnu/libEGL.so
2.8. if you have this error: The imported target "Qt5::Gui" references the file, "/usr/lib/x86_64-linux-gnu/libGL.so"
sudo rm /usr/lib/x86_64-linux-gnu/libGL.so; sudo ln /usr/lib/x86_64-linux-gnu/libGL.so.1 /usr/lib/x86_64-linux-gnu/libGL.so
2.9. If you have this error: CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
CUDA_nppi_LIBRARY (ADVANCED)
$ cd ~/workspace/opencv-2.4.13.6/cmake
$ cp ../../opencv-3.4.4/cmake/FindCUDA.cmake ./
$ cp ../../opencv-3.4.4/cmake/OpenCVDetectCUDA.cmake ./
$ cd ../build
$ make all -j32
2.10. If you have this error, opencv_dep_CUDA_nppi_LIBRARY is not found:-
replace the root > cmake > FindCUDA.cmake from opencv3.4.4 to 2.4.13.6
2.11. If you have this error, "gflags-2.1.1/src/gflags.cc:90:20: fatal error: config.h: No such file or directory"
cmake version is too new, need to downgrade
3. create a conda virtual env named as caffe2 with python2.7, install:
$ pip install scikit-image
$ pip install protobuf (after completed step 2)
4. download caffe and build it with 'cmake ..' and 'make -j$(nproc)' and 'sudo make install'
$ cd caffe/
$ cd build/
$ sudo rm -r ./*
$ cmake .. -DCMAKE_INSTALL_PREFIX:PATH=/home/ninja/.conda/envs/caffe
$ make -j32
$ sudo make install
$ make runtest
$ make pycaffe
4.1 if sudo make install failed due to undefined reference to `ZSTD_initCStream, run:
$ conda list
$ conda install libtiff (it should install zstd)
or you can just modify the library path by pointing to system libraries only
$ gedit ~/.bashrc
>> # insert into last line
>> export $LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu
$ source ~/.bashrc
$ conda activate caffe2(compile again, make -j32)
4.2 C++ exception with description "locale::facet::_S_create_c_locale name not valid" thrown in SetUp().
$ export LC_ALL="C"
(rerun make runtest)
4.3 If you received the following error:-
CXX/LD -o .build_release/tools/convert_imageset.bin .build_release/lib/libcaffe.so: undefined reference to `google::protobuf::io::CodedOutputStream::WriteVarint64ToArray(unsigned long long, unsigned char*)' .build_release/lib/libcaffe.so: undefined reference to `google::protobuf::internal::WireFormatLite::WriteInt64(int, long long, google::protobuf::io::CodedOutputStream*)' .build_release/lib/libcaffe.so: undefined reference to `google::protobuf::UnknownFieldSet::AddVarint(int, unsigned long long)' .build_release/lib/libcaffe.so: undefined reference to `google::protobuf::io::CodedOutputStream::VarintSize64(unsigned long long)' .build_release/lib/libcaffe.so: undefined reference to `google::protobuf::io::CodedOutputStream::WriteVarint64(unsigned long long)' collect2: error: ld returned 1 exit status make: *** [.build_release/tools/convert_imageset.bin] Error 1
$ ldd .build_release/lib/libcaffe.so
(check libcaffe.so is using which library, it should point to
libprotobuf.so.11 => /usr/local/lib/libprotobuf.so.11. If it links to
library under anaconda, just delete it.)
5.If all succeed, you need to set the pythonpath as:-
$ export PYTHONPATH=/home/ninja/.conda/envs/caffe/python:$PYTHONPATH
6. run python and then import caffe, you may need to install protobuf as pip install protobuf==3.1.0 and pip install scikit-image (check using protoc --version)
7. /usr/bin/ld: cannot find -llib64
check the CUDNN_LIBRARY or CUDA_LIBRARY path, should be full path to the specific library, /home/ubuntu/workspace/cuda/lib64/libcudnn.so, not folder name /home/ubuntu/workspace/cuda/lib64/
8. /home/ubuntu/workspace/caffe-opencv4.2-cuda10.0/include/caffe/proto/caffe.pb.h:17:2: error: #error This file was generated by an older version of protoc which is
or
./include/caffe/util/cudnn.hpp:8:34: fatal error: caffe/proto/caffe.pb.h: No such file or directory
$ protoc src/caffe/proto/caffe.proto --cpp_out=.
$ mkdir include/caffe/proto
$ mv src/caffe/proto/caffe.pb.h include/caffe/proto9. how to force cmake to use one particular cuda architecture?
-DCUDA_ARCH_NAME=Manual -DCUDA_ARCH_BIN="75" -DCUDA_ARCH_PTX="75"
Example of .bashrc for building opencv
PATH=$PATH:$HOME/anaconda/bin:/usr/local/cuda/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib:/usr/lib/x86_64-linux-gnu/:/home/ninja/anaconda3/lib
Example of .bashrc for building caffe
export PYTHONPATH=/home/ninja/.conda/envs/caffe/python:$PYTHONPATH
PATH=$PATH:$HOME/anaconda/bin:/usr/local/cuda/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib:/usr/lib/x86_64-linux-gnu/:/home/ninja/.conda/envs/caffe/lib
10. (-218:No OpenGL support) Library was built without OpenGL support in function 'CvWindow'：-
-D OpenGL_GL_PREFERENCE=GLVND \
-D WITH_QT=ON \
-D WITH_OPENGL=ON \
-D WITH_GTK_2_X=ON \
11. How to find the correct path of opencv from a new C++ project:
During opencv compilation, set the install path in cmakelists.txt:
-D CMAKE_INSTALL_PREFIX=/home/peseyes/Documents/opencv-4.4.0/distribute \
During the new project compilation, set the opencv path in cmakelists.txt:-
SET(OpenCV_DIR /home/peseyes/Documents/opencv-4.4.0/distribute/share/opencv4) find_package(OpenCV 4 REQUIRED)
Sunday, 14 April 2019
transfer file between windows and ubuntu - filezilla
sometime it is time consuming to upload the data or file using mobixterm
may be you can try use filezilla and following are the settings:-
host: sftp://10.80.43.31
username: name
password: word:
port: 22
(image source: https://filezilla.net/)
may be you can try use filezilla and following are the settings:-
host: sftp://10.80.43.31
username: name
password: word:
port: 22
sudo apt update - proxy
how to use proxy in ubuntu?
here are few commands that might help
env | grep proxy
unset no_proxy
sudo nano /etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
http_proxy="http://proxy.company.com:port/"
https_proxy="https://proxy.company.com:port/"
ftp_proxy="ftp://proxy.company.com:port/"
env | grep proxy
sudo apt update
sudo apt upgrade
sudo nano /etc/apt/sources.list (make sure use the default settings from another computer)
here are few commands that might help
env | grep proxy
unset no_proxy
sudo nano /etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
http_proxy="http://proxy.company.com:port/"
https_proxy="https://proxy.company.com:port/"
ftp_proxy="ftp://proxy.company.com:port/"
env | grep proxy
sudo apt update
sudo apt upgrade
sudo nano /etc/apt/sources.list (make sure use the default settings from another computer)
Thursday, 7 March 2019
pyinstaller
In order to convert the python code to exe in windows, we can use pyinstaller.
Here are the steps:
1. Make sure you can run the python code under windows.
If you are running with cuda, make sure the device id is selected correctly, 0/1.
2. Once you can run the python code in windows, next is to install the pyinstaller.
It must be run under python > 3.3 and you can install it as following:
$ pip install pyinstaller
3. Once you have the pyinstaller, next you need to setup a main.spec.
Assume your python code is main.py.
You can create a main.spec file as following:
Run the following command to convert main.py into main.exe.
Output will be generated in two folders, build and dist.
You only need to look at the dist folder > main.exe.
$ pyinstaller main.spec
5. During compilation, you may receive some errors.
Most probably is due to the missing packages under python.
For example, I need to run the following code to install some packages.
$ pip install setuptools --upgrade
$ pip uninstall six
$ pip install six
$ conda install --channel https://conda.anaconda.org/bpentz pyqt5
You can navigate to root > dist > main.exe.
If you running the main.py with some additional files, remember to copy into dist folder.
You may run the exe in cmd to see the errors.
p/s: I remembered the compiled exe cannot be used for commercial purpose as it uses some Microsoft dll or licensed dll, please be aware.
Here are the steps:
1. Make sure you can run the python code under windows.
If you are running with cuda, make sure the device id is selected correctly, 0/1.
2. Once you can run the python code in windows, next is to install the pyinstaller.
It must be run under python > 3.3 and you can install it as following:
$ pip install pyinstaller
3. Once you have the pyinstaller, next you need to setup a main.spec.
Assume your python code is main.py.
You can create a main.spec file as following:
# -*- mode: python -*-
#filename = Main.spec
import sys
myLibPath = './libs'
sys.path.append(myLibPath)
a = Analysis(['Main.py'],
pathex=['/Users/.../ProgramNameDir'],
hiddenimports=[],
hookspath=None,
runtime_hooks=None)
pyz = PYZ(a.pure)
exe = EXE(pyz,
Tree('img',prefix='img'),
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
name='Main',
debug=False,
strip=None,
upx=True,
console=False , icon='mainIcon.ico')
app = BUNDLE(exe,
name='Main.app',
info_plist={ 'NSHighResolutionCapable': 'True'}, #<-- Option for High Resolution
icon='mainIcon.ico')
4. Once you setup the main.spec, then you are all set and ready to compile it.
Run the following command to convert main.py into main.exe.
Output will be generated in two folders, build and dist.
You only need to look at the dist folder > main.exe.
$ pyinstaller main.spec
5. During compilation, you may receive some errors.
Most probably is due to the missing packages under python.
For example, I need to run the following code to install some packages.
$ pip install setuptools --upgrade
$ pip uninstall six
$ pip install six
$ conda install --channel https://conda.anaconda.org/bpentz pyqt5
6. Once compilation completed, congratulations, you are done.
You can navigate to root > dist > main.exe.
If you running the main.py with some additional files, remember to copy into dist folder.
You may run the exe in cmd to see the errors.
p/s: I remembered the compiled exe cannot be used for commercial purpose as it uses some Microsoft dll or licensed dll, please be aware.
Subscribe to:
Posts (Atom)