ldd /usr/local/lib/libopencv_core.so
Thursday, 1 August 2019
Thursday, 25 July 2019
cmake Could NOT find Atlas
$ sudo apt install
$ sudo apt install
$ sudo apt install
liblapacke-dev$ sudo apt install
libatlas-base-dev$ sudo apt install
libatlas3-baseThursday, 18 July 2019
THCudaCheck FAIL file=/pytorch/aten/src/THC/THCGeneral.cpp line=405 error=11 : invalid argument
Check the pytorch compatible version with cuda10.0.
You can install torch 1.0.0 as:
$ pip install -U https://download.pytorch.org/whl/cu100/torch-1.0.0-cp36-cp36m-linux_x86_64.whl
$
Or you can install torch 1.1.0 as:
$ pip install -U https://download.pytorch.org/whl/cu100/torch-1.1.0-cp36-cp36m-linux_x86_64.whl
$
You can install torch 1.0.0 as:
$ pip install -U https://download.pytorch.org/whl/cu100/torch-1.0.0-cp36-cp36m-linux_x86_64.whl
$
pip install https://download.pytorch.org/whl/cu100/torchvision-0.2.2-cp36-cp36m-linux_x86_64.whl Or you can install torch 1.1.0 as:
$ pip install -U https://download.pytorch.org/whl/cu100/torch-1.1.0-cp36-cp36m-linux_x86_64.whl
$
pip install --force https://download.pytorch.org/whl/cu100/torchvision-0.3.0-cp36-cp36m-linux_x86_64.whl
Sunday, 7 July 2019
how to build caffe in ubuntu16, cuda10.0, cudnn7.6, python3.5
This is similar to the previous post where we change the focus to python3.5.
Make sure you have install caffe for python2.7 before continue reading this post.
Here are few steps I have done for successfully compiling the caffe for python 3.5.
1. Clone the default caffe folder into a new folder named as caffe3. Then, delete all files in the build folder,
$ cp ./caffe/* ./caffe3/
$ sudo rm -r ./caffe3/build/*
2. Create a new conda environment and I named it as caffe3 as well. Make sure you use python3.5 as following,
$ conda create -n caffe3 python=3.5
3. Activate the caffe3 env and then install the numpy as following,
$ conda activate caffe3
$ pip install numpy
$ pip install scikit-image
4. Install additional dependencies as following:
5. Once completed, cd to the caffe3 root folder and copy the makefile as following:
$ cd ./caffe3
$
6. You will see a new file named as Makefile.config under ./caffe folder. Copy and paste the Makefile content as given in the appendix below. Note that please change the path accordingly as my anaconda env was installed at /home/ninja/.conda/envs/*, yours might be different such as /home/ninja/anaconda3/envs/*.
7. Once everything is ready, you are good to compile caffe, make sure you are in the ./caffe3 folder:
$ make clean
$ make all -j32
$ make test
$ make runtest
$ make pycaffe
$ make distribute
(you should be able to see the distribute folder under caffe3)
8.If you have no error in compilation, congratulation! Next is to set the path in ~/.bashrc and you should able to import caffe.
$ sudo gedit ~/.bashrc
then add the following lines into ~/.bashrc, save it and source it
export PYTHONPATH=/home/ninja/caffe3/distribute/python:$PYTHONPATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib:/usr/lib/x86_64-linux-gnu/:/home/ninja/caffe3/distribute/lib
$ source ~/.bashrc
$ conda activate caffe3
$ python (version 3.5.*)
>>> import caffe
>>> (pray, no error)
Example1 of Error:
./include/caffe/util/signal_handler.h:4:34: fatal error: caffe/proto/caffe.pb.h: No such file or directory
If you are using conda, you need to include the env lib into the LD_LIBRARY_PATH:
$ export LD_LIBRARY_PATH=/home/ninja/.conda/envs/caffe3/lib
Example2 of Error:
xxx... .nccl.h: No such file or directory
If you are using conda, you need to copy all include and lib files into env:
$ cp nccl_2.4.7-1+cuda10.0_x86_64/include/nccl* /home/ninja/.conda/envs/c3d11/include/
$ cp nccl_2.4.7-1+cuda10.0_x86_64/lib/libnccl* /home/ninja/.conda/envs/c3d11/lib/
Example3 of Error:
xxx... .cudnn.h: No such file or directory
If you are using conda, you need to copy all include and lib files into env:
$ cp cuda/include/cudnn.h /home/ninja/.conda/envs/c3d11/include/
$ cp cuda/lib64/libcudnn* /home/ninja/.conda/envs/c3d11/lib/
I am using cudnn-10.0-linux-x64-v7.6.0.64.
Example4 of Error:
/usr/include/boost/python/detail/wrap_python.hpp:50:23: fatal error: pyconfig.h: No such file or directory
$ locate pyconfig.h
>> set the path in Makefile.config > PYTHON_INCLUDE
Example5 of Error:
Solve the problem: "cannot find -lboost_python3" when using Python3 Ubuntu16.04
$ cd /usr/lib/x86_64-linux-gnu
$ sudo ln -s libboost_python-py35.so libboost_python3.so
Example of Makefile, CUDA 8.0, CUDNN 6.0 used in this post.
Example of Makefile, CUDA 10.0, CUDNN 7.6 used in this post.
Make sure you have install caffe for python2.7 before continue reading this post.
Here are few steps I have done for successfully compiling the caffe for python 3.5.
1. Clone the default caffe folder into a new folder named as caffe3. Then, delete all files in the build folder,
$ cp ./caffe/* ./caffe3/
$ sudo rm -r ./caffe3/build/*
2. Create a new conda environment and I named it as caffe3 as well. Make sure you use python3.5 as following,
$ conda create -n caffe3 python=3.5
3. Activate the caffe3 env and then install the numpy as following,
$ conda activate caffe3
$ pip install numpy
$ pip install scikit-image
4. Install additional dependencies as following:
$ sudo apt-get install python3-skimage
$ sudo apt-get install python3-protobuf5. Once completed, cd to the caffe3 root folder and copy the makefile as following:
$ cd ./caffe3
$
cp Makefile.config.example Makefile.config6. You will see a new file named as Makefile.config under ./caffe folder. Copy and paste the Makefile content as given in the appendix below. Note that please change the path accordingly as my anaconda env was installed at /home/ninja/.conda/envs/*, yours might be different such as /home/ninja/anaconda3/envs/*.
7. Once everything is ready, you are good to compile caffe, make sure you are in the ./caffe3 folder:
$ make clean
$ make all -j32
$ make test
$ make runtest
$ make pycaffe
$ make distribute
(you should be able to see the distribute folder under caffe3)
8.If you have no error in compilation, congratulation! Next is to set the path in ~/.bashrc and you should able to import caffe.
$ sudo gedit ~/.bashrc
then add the following lines into ~/.bashrc, save it and source it
export PYTHONPATH=/home/ninja/caffe3/distribute/python:$PYTHONPATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib:/usr/lib/x86_64-linux-gnu/:/home/ninja/caffe3/distribute/lib
$ source ~/.bashrc
$ conda activate caffe3
$ python (version 3.5.*)
>>> import caffe
>>> (pray, no error)
Example1 of Error:
./include/caffe/util/signal_handler.h:4:34: fatal error: caffe/proto/caffe.pb.h: No such file or directory
If you are using conda, you need to include the env lib into the LD_LIBRARY_PATH:
$ export LD_LIBRARY_PATH=/home/ninja/.conda/envs/caffe3/lib
Example2 of Error:
xxx... .nccl.h: No such file or directory
If you are using conda, you need to copy all include and lib files into env:
$ cp nccl_2.4.7-1+cuda10.0_x86_64/include/nccl* /home/ninja/.conda/envs/c3d11/include/
$ cp nccl_2.4.7-1+cuda10.0_x86_64/lib/libnccl* /home/ninja/.conda/envs/c3d11/lib/
Example3 of Error:
xxx... .cudnn.h: No such file or directory
If you are using conda, you need to copy all include and lib files into env:
$ cp cuda/include/cudnn.h /home/ninja/.conda/envs/c3d11/include/
$ cp cuda/lib64/libcudnn* /home/ninja/.conda/envs/c3d11/lib/
I am using cudnn-10.0-linux-x64-v7.6.0.64.
Example4 of Error:
/usr/include/boost/python/detail/wrap_python.hpp:50:23: fatal error: pyconfig.h: No such file or directory
$ locate pyconfig.h
>> set the path in Makefile.config > PYTHON_INCLUDE
Example5 of Error:
Solve the problem: "cannot find -lboost_python3" when using Python3 Ubuntu16.04
$ cd /usr/lib/x86_64-linux-gnu
$ sudo ln -s libboost_python-py35.so libboost_python3.so
Example6 of Error:
Solve the problem: "/usr/bin/ld: cannot find -lopencv_imgcodecs" when using Python3 Ubuntu16.04, modify Makefile.config:-
Solve the problem: "/usr/bin/ld: cannot find -lopencv_imgcodecs" when using Python3 Ubuntu16.04, modify Makefile.config:-
PYTHON_LIBRARIES := boost_python3 python3.5m opencv_imgcodecs
PYTHON_LIB := $(ANACONDA_HOME)/lib /home/ninja/anaconda3/envs/caffe3/lib /home/ninja/workspace/opencv-3.4.4/distribute/lib
## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!
LINKFLAGS := -Wl,-rpath,$(HOME)/anaconda3/lib
# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1
# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1
# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 0
# USE_LEVELDB := 0
# USE_LMDB := 0
# This code is taken from https://github.com/sh1r0/caffe-android-lib
# USE_HDF5 := 0
# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
# You should not set this flag if you will be reading LMDBs with any
# possibility of simultaneous read and write
ALLOW_LMDB_NOLOCK := 1
# Uncomment if you're using OpenCV 3
OPENCV_VERSION := 3
# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++
# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda-8.0
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr
# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility.
# For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility.
# For CUDA >= 9.0, comment the *_20 and *_21 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_60,code=sm_61
# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas
# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib
# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app
# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
# PYTHON_INCLUDE := /usr/include/python2.7 \
/usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
ANACONDA_HOME := /home/ninja/.conda/envs/caffe3
PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
$(ANACONDA_HOME)/include/python3.5 \
$(ANACONDA_HOME)/lib/python3.5/site-packages/numpy/core/include
# Uncomment to use Python 3 (default is Python 2)
PYTHON_LIBRARIES := boost_python3 python3.5m
PYTHON_INCLUDE := /home/ninja/.conda/envs/caffe3/include/python3.5m \
/home/ninja/.conda/envs/caffe3/lib/python3.5/site-packages/numpy/core/include
# We need to be able to find libpythonX.X.so or .dylib.
# PYTHON_LIB := /usr/lib
PYTHON_LIB := $(ANACONDA_HOME)/lib /home/ninja/.conda/envs/caffe3/lib
# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib
# Uncomment to support layers written in Python (will link against Python libs)
WITH_PYTHON_LAYER := 1
# Whatever else you find you need goes here.
#INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
#LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial /home/ninja/opencv/include/ /home/ninja/.conda/envs/caffe3/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial /home/ninja/opencv/build/lib
# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib
# NCCL acceleration switch (uncomment to build with NCCL)
# https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0)
USE_NCCL := 1
# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
USE_PKG_CONFIG := 1
# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute
# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1
# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0
# enable pretty build (comment to see full commands)
Q ?= @
Example of Makefile, CUDA 10.0, CUDNN 7.6 used in this post.
## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!
LINKFLAGS := -Wl,-rpath,$(HOME)/anaconda3/lib
# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1
# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1
# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 0
# USE_LEVELDB := 0
# USE_LMDB := 0
# This code is taken from https://github.com/sh1r0/caffe-android-lib
# USE_HDF5 := 0
# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
# You should not set this flag if you will be reading LMDBs with any
# possibility of simultaneous read and write
ALLOW_LMDB_NOLOCK := 1
# Uncomment if you're using OpenCV 3
OPENCV_VERSION := 3
# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++
# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda-10.0
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr
# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility.
# For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility.
# For CUDA >= 9.0, comment the *_20 and *_21 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_75,code=sm_75
# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas
# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib
# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app
# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
# PYTHON_INCLUDE := /usr/include/python2.7 \
/usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
ANACONDA_HOME := /home/ninja/.conda/envs/caffe3
PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
$(ANACONDA_HOME)/include/python3.5 \
$(ANACONDA_HOME)/lib/python3.5/site-packages/numpy/core/include
# Uncomment to use Python 3 (default is Python 2)
PYTHON_LIBRARIES := boost_python3 python3.5m
PYTHON_INCLUDE := /home/ninja/.conda/envs/caffe3/include/python3.5m \
/home/ninja/.conda/envs/caffe3/lib/python3.5/site-packages/numpy/core/include
# We need to be able to find libpythonX.X.so or .dylib.
# PYTHON_LIB := /usr/lib
PYTHON_LIB := $(ANACONDA_HOME)/lib /home/ninja/.conda/envs/caffe3/lib
# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib
# Uncomment to support layers written in Python (will link against Python libs)
WITH_PYTHON_LAYER := 1
# Whatever else you find you need goes here.
#INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
#LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial /home/ninja/opencv/include/ /home/ninja/.conda/envs/caffe3/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial /home/ninja/opencv/build/lib
# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib
# NCCL acceleration switch (uncomment to build with NCCL)
# https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0)
USE_NCCL := 1
# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
USE_PKG_CONFIG := 1
# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute
# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1
# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0
# enable pretty build (comment to see full commands)
Q ?= @
Thursday, 27 June 2019
how to install bazel in ubuntu?
standard version:-
$ echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
$ curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add -
$ sudo apt-get update && sudo apt-get install bazel
$ sudo apt-get upgrade bazel
customized version:-
$ echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
$ curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add -
$ sudo apt-get update && sudo apt-get install bazel
$ sudo apt-get upgrade bazel
customized version:-
$ BAZEL_VERSION="0.26.0" # insert your desired version here, for example 0.26.0
$ wget https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh # if not on x86_64, change that too
$ chmod +x bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh # or the file you just downloaded
$ ./bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh --user
$ bazel version # this should now print the same as BAZEL_VERSION
Wednesday, 19 June 2019
Ubuntu How to Find *.jpg in Current Folder and Subfolders
In general, here are few useful commands:
1. find all images in current and its subfolders:-
find . -name "*.jpg" > temp.txt
2. To list all images with full path:-
find $(pwd) -name "*.ppm" > temp.txt
2. To list all images with full path:-
find $(pwd) -name "*.ppm" > temp.txt
3. find all images in current and its subfolders, then count it
find . -name "*.jpg"| wc -l
=== Appendix ===
How to find files and directories in Linux
(ps: copy from https://www.computerhope.com/issues/ch001723.htm)
In Linux operating systems, the find
command may be used to search for files and directories on your
computer. To proceed, select a link from the following list or go
through each section in order.
Basic functionality of find
Running find without any options will produce a list of every file and directory in and beneath the working directory. For instance, if your working directory is /home/hope/Documents, running find will output the following:
- Every file in /home/hope/Documents.
- Every subdirectory in /home/hope/Documents.
- Every file in each of those subdirectories.
Let's see it in action. First, let's check our working directory by using the pwd command:
pwd
/home/hope/Documents
Now let's run find without any options:
find
. ./images ./images/hp ./images/hp/snape.jpg ./images/hp/harry.jpg ./images/memes ./images/memes/winteriscoming.jpg ./images/memes/goodguygary.JPG ./images/memes/picard.jpg ./gimp-2.8.16.tar.bz2 ./hp-fanfic ./hp-fanfic/malfoys-revenge.doc ./hp-fanfic/weekend-at-hagreds.doc ./hp-fanfic/dumbledores-lament.doc ./archlinux-2016.02.01-dual.iso
In this example, we see a total of ten files and four subdirectories in and beneath our Documents folder.
Notice that the output starts with a single dot, which represents the working directory. Running find with no options is the same as specifying that the search should begin in the working directory, like this:
find .
The above example is the "proper" way to use find. If you try to use it on another UNIX-like operating system, such as FreeBSD, you will find that specifying a directory is required, so it's good practice to use this form of the command.
Specifying where to search
To only list files and subdirectories that are contained in the directory /home/hope/Documents/images, specify it as the first argument of the command:
find /home/hope/Documents/images
/home/hope/Documents/images /home/hope/Documents/images/hp /home/hope/Documents/images/hp/snape.jpg /home/hope/Documents/images/hp/harry.jpg /home/hope/Documents/images/memes /home/hope/Documents/images/memes/winteriscoming.jpg /home/hope/Documents/images/memes/goodguygary.JPG /home/hope/Documents/images/memes/picard.jpg
Notice that the full path is also shown in the results.
If our working directory is /home/hope/Documents, we can use the following command, which finds the same files:
find ./images
But this time, the output reflects the starting location of the search and looks like this:
./images ./images/hp ./images/hp/snape.jpg ./images/hp/harry.jpg ./images/memes ./images/memes/winteriscoming.jpg ./images/memes/goodguygary.JPG ./images/memes/picard.jpg
By default, the search will look in every subdirectory of
your starting location. If you want to restrict how many levels of
subdirectory to search, you can use the -maxdepth option with a number.
For instance, specifying -maxdepth 1 will search
only in the directory where the search begins. If any subdirectories are
found, they will be listed, but not searched.
find . -maxdepth 1
. ./images ./bigfiles.txt ./gimp-2.8.16.tar.bz2 ./hp-fanfic ./archlinux-2016.02.01-dual.iso
Specifying -maxdepth 2 will search the directory and one subdirectory deep:
find . -maxdepth 2
. ./images ./images/hp ./images/memes ./gimp-2.8.16.tar.bz2 ./hp-fanfic ./hp-fanfic/malfoys-revenge.doc ./hp-fanfic/weekend-at-hagreds.doc ./hp-fanfic/dumbledores-lament.doc ./archlinux-2016.02.01-dual.iso
Specifying -maxdepth 3 will search one level deeper than that:
find . -maxdepth 3
. ./images ./images/hp ./images/hp/snape.jpg ./images/hp/harry.jpg ./images/memes ./images/memes/winteriscoming.jpg ./images/memes/goodguygary.JPG ./images/memes/picard.jpg ./gimp-2.8.16.tar.bz2 ./hp-fanfic ./hp-fanfic/malfoys-revenge.doc ./hp-fanfic/weekend-at-hagreds.doc ./hp-fanfic/dumbledores-lament.doc ./archlinux-2016.02.01-dual.iso
Finding by name
To restrict your search results to match only files and directories that have a certain name, use the -name option and put the name in quotes:
find . -name "picard.jpg"
./images/memes/picard.jpg
You can also use wildcards as part of your file name. For instance, to find all files whose name ends in .jpg, you can use an asterisk to represent the rest of the file name. When you run the command, the shell will glob the file name into anything that matches the pattern:
find . -name "*.jpg"
./images/hp/snape.jpg ./images/hp/harry.jpg ./images/memes/winteriscoming.jpg ./images/memes/picard.jpg
Notice that our command didn't list the file whose extension (in this case, JPG) is in capital letters. That's because unlike other operating systems, such as Microsoft Windows, Linux file names are case-sensitive.
To perform a case-insensitive search instead, use the -iname option:
find . -iname "*.jpg"
./images/hp/snape.jpg ./images/hp/harry.jpg ./images/memes/winteriscoming.jpg ./images/memes/goodguygary.JPG ./images/memes/picard.jpg
Finding only files, or only directories
To list files only and omit directory names from your results, specify -type f:
find . -type f
./images/hp/snape.jpg ./images/hp/harry.jpg ./images/memes/winteriscoming.jpg ./images/memes/goodguygary.JPG ./images/memes/picard.jpg ./gimp-2.8.16.tar.bz2 ./hp-fanfic/malfoys-revenge.doc ./hp-fanfic/weekend-at-hagreds.doc ./hp-fanfic/dumbledores-lament.doc ./archlinux-2016.02.01-dual.iso
To list directories only and omit file names, specify -type d:
find . -type d
. ./images ./images/hp ./images/memes ./hp-fanfic
Finding files based on size
To display only files of a certain size, you can use the -size
option. To specify the size, use a plus or a minus sign (for "more
than" or "less than"), a number, and a quantitative suffix such as k, M, or G.
For instance, to find files that are "bigger than 50 kilobytes", use -size +50k:
find . -size +50k
./images/memes/winteriscoming.jpg ./gimp-2.8.16.tar.bz2 ./archlinux-2016.02.01-dual.iso
For files "bigger than 10 megabytes", use -size +10M:
find . -size +10M
./gimp-2.8.16.tar.bz2 ./archlinux-2016.02.01-dual.iso
For "bigger than 1 gigabyte", use -size +1G:
find . -size +1G
./archlinux-2016.02.01-dual.iso
For files in a certain size range, use two -size options. For instance, to find files "bigger than 10 megabytes, but smaller than 1 gigabyte", specify -size +10M -size -1G:
find . -size +10M -size -1G
./gimp-2.8.16.tar.bz2
Finding files based on modification, access, or status change
The -mtime option restricts search by how many
days since the file's contents were modified. To specify days in the
past, use a negative number. For example, to find only those files which
were modified in the past two days (48 hours ago), use -mtime -2:
find . -mtime -2
The -mmin option does the same thing, but in terms
of minutes, not days. For instance, this command shows only files
modified in the past half hour:
find . -mmin -30
A similar option is -ctime, which checks when a file's status was last changed, measured in days. A status change is a change in the file's metadata. For instance, changing the permissions of a file is status change.
The option -cmin will search for a status change, measured in minutes.
You can also search for when a file was last accessed — in other words, when its contents were most recently viewed. The -atime option is used to search for files based upon their most recent access time, measured in days.
The -amin option will perform the same search restriction, but measured in minutes.
Redirecting output to a text file
If you are performing a very large search, you may want
to save your search results in a file, so that you can view the results
later. You can do this by redirecting your find output to a file:
find . -iname "*.jpg" > images.txt
You can then open your results in a text editor, or print them with the cat command.
cat images.txt
./images/hp/snape.jpg ./images/hp/harry.jpg ./images/memes/winteriscoming.jpg ./images/memes/goodguygary.JPG ./images/memes/picard.jpg
Alternatively, you can pipe your output to the tee command, which will print the output to the screen and write it to a file:
find . -size +500M | tee bigfiles.txt
./archlinux-2016.02.01-dual.iso
cat bigfiles.txt
./archlinux-2016.02.01-dual.iso
Suppressing error messages
You may receive the error message "Permission denied" when performing a search. For instance, if you search the root directory as a normal user:
find /
find: `/var/lib/sudo/ts': Permission denied find: `/var/lib/sudo/lectured': Permission denied find: `/var/lib/polkit-1': Permission denied find: `/var/lib/container': Permission denied find: `/var/lib/gdm3/.dbus': Permission denied find: `/var/lib/gdm3/.config/ibus': Permission denied ...
You will receive that error message if find tries to access a file that your user account doesn't have permission to read. You may be able to perform the search as the superuser (root), which has complete access to every file on the system. But it's not recommended to do things as root, unless there are no other options.
If all you need to do is hide the "Permission denied" messages, you can add 2&>1 | grep -v "Permission denied" to the end of your command, like this:
find / 2>&1 | grep -v "Permission denied"
The above example filters out the "Permission denied" messages from your search. How?
2>&1 is a special redirect that sends error messages to the standard output to pipe the combined lines of output to the grep command. grep -v then performs an inverse match on "Permission denied", displaying only lines which do not contain that string.
Redirecting and using grep to filter the error messages
is a useful technique when "Permission denied" is cluttering your search
results and you can't perform the search as root.
Examples
find ~/. -name "*.txt" -amin -120
Find all files in your home directory and below which end in the extension ".txt". Display only files accessed in the past two hours.
find . -name "*.zip" -size +10M -mtime -3
Find all files in the working directory and below whose name has the extension ".zip" and whose size is greater than 10 megabytes. Display only files whose contents were modified in the last 72 hours.
find . -iname "*report*" -type f -maxdepth 2
Perform a case-insensitive search for files that contain the word "report"
in their name. If the search finds a directory with "report" in its
name, do not display it. Search only in the working directory, and one
directory level beneath it.
find / -name "*init*" 2>&1 | grep -v "Permission denied" | tee ~/initfiles.txt
Find all files on the system whose name contains "init", suppressing error messages. Display results on the screen and output them to a file in your home directory named "initfiles.txt".
Subscribe to:
Posts (Atom)