dozob

Reference:

https://www.linuxfromscratch.org/blfs/view/svn/general/libjpeg.html

$ sudo apt install cmake

Download

https://downloads.sourceforge.net/libjpeg-turbo/libjpeg-turbo-2.1.4.tar.gz

$ wget https://downloads.sourceforge.net/libjpeg-turbo/libjpeg-turbo-2.1.4.tar.gz
$ tar -xvf libjpeg-turbo-2.1.4.tar.gz
$ cd libjpeg-turbo-2.1.4/

$ mkdir build
$ cd build

$ cmake -DCMAKE_INSTALL_PREFIX=/usr \
        -DCMAKE_BUILD_TYPE=RELEASE  \
        -DENABLE_STATIC=FALSE       \
        -DCMAKE_INSTALL_DOCDIR=/usr/share/doc/libjpeg-turbo-2.1.4 \
        -DCMAKE_INSTALL_DEFAULT_LIBDIR=lib  \
        ..

$ make

$ sudo make install
...
Install the project...
-- Install configuration: "RELEASE"
-- Installing: /usr/lib/libturbojpeg.so.0.2.0
-- Installing: /usr/lib/libturbojpeg.so.0
-- Set runtime path of "/usr/lib/libturbojpeg.so.0.2.0" to "/usr/lib"
-- Installing: /usr/lib/libturbojpeg.so
-- Installing: /usr/bin/tjbench
-- Set runtime path of "/usr/bin/tjbench" to "/usr/lib"
-- Installing: /usr/include/turbojpeg.h
-- Installing: /usr/bin/rdjpgcom
-- Set runtime path of "/usr/bin/rdjpgcom" to "/usr/lib"
-- Installing: /usr/bin/wrjpgcom
-- Set runtime path of "/usr/bin/wrjpgcom" to "/usr/lib"
-- Installing: /usr/share/doc/libjpeg-turbo-2.1.4/README.ijg
-- Installing: /usr/share/doc/libjpeg-turbo-2.1.4/README.md
-- Installing: /usr/share/doc/libjpeg-turbo-2.1.4/example.txt
-- Installing: /usr/share/doc/libjpeg-turbo-2.1.4/tjexample.c
-- Installing: /usr/share/doc/libjpeg-turbo-2.1.4/libjpeg.txt
-- Installing: /usr/share/doc/libjpeg-turbo-2.1.4/structure.txt
-- Installing: /usr/share/doc/libjpeg-turbo-2.1.4/usage.txt
-- Installing: /usr/share/doc/libjpeg-turbo-2.1.4/wizard.txt
-- Installing: /usr/share/doc/libjpeg-turbo-2.1.4/LICENSE.md
-- Installing: /usr/share/man/man1/cjpeg.1
-- Installing: /usr/share/man/man1/djpeg.1
-- Installing: /usr/share/man/man1/jpegtran.1
-- Installing: /usr/share/man/man1/rdjpgcom.1
-- Installing: /usr/share/man/man1/wrjpgcom.1
-- Installing: /usr/lib/pkgconfig/libjpeg.pc
-- Installing: /usr/lib/pkgconfig/libturbojpeg.pc
-- Installing: /usr/lib/cmake/libjpeg-turbo/libjpeg-turboConfig.cmake
-- Installing: /usr/lib/cmake/libjpeg-turbo/libjpeg-turboConfigVersion.cmake
-- Installing: /usr/lib/cmake/libjpeg-turbo/libjpeg-turboTargets.cmake
-- Installing: /usr/lib/cmake/libjpeg-turbo/libjpeg-turboTargets-release.cmake
-- Installing: /usr/include/jconfig.h
-- Installing: /usr/include/jerror.h
-- Installing: /usr/include/jmorecfg.h
-- Installing: /usr/include/jpeglib.h
-- Installing: /usr/lib/libjpeg.so.62.3.0
-- Installing: /usr/lib/libjpeg.so.62
-- Set runtime path of "/usr/lib/libjpeg.so.62.3.0" to "/usr/lib"
-- Installing: /usr/lib/libjpeg.so
-- Installing: /usr/bin/cjpeg
-- Set runtime path of "/usr/bin/cjpeg" to "/usr/lib"
-- Installing: /usr/bin/djpeg
-- Set runtime path of "/usr/bin/djpeg" to "/usr/lib"
-- Installing: /usr/bin/jpegtran
-- Set runtime path of "/usr/bin/jpegtran" to "/usr/lib"

$ whereis *jpeg*
cjpeg: /usr/bin/cjpeg /usr/share/man/man1/cjpeg.1
djpeg: /usr/bin/djpeg /usr/share/man/man1/djpeg.1
jpegtran: /usr/bin/jpegtran /usr/share/man/man1/jpegtran.1
libjpeg: /usr/lib/libjpeg.so
libjpeg: /usr/lib/libjpeg.so
libjpeg.so: /usr/lib/x86_64-linux-gnu/libjpeg.so.8 /usr/lib/libjpeg.so.62 /usr/lib/libjpeg.so
libjpeg.so.62.3: /usr/lib/libjpeg.so.62.3.0
libturbojpeg: /usr/lib/libturbojpeg.so
libturbojpeg.so: /usr/lib/libturbojpeg.so /usr/lib/libturbojpeg.so.0
libturbojpeg.so.0.2: /usr/lib/libturbojpeg.so.0.2.0

0. Remove CUDA files

$ sudo apt-get remove --purge '^nvidia-.*'

$ sudo apt-get remove --purge 'cuda*'
$ sudo apt-get autoremove --purge 'cuda*'

$ sudo rm -rf /usr/local/cuda
$ sudo rm -rf /usr/local/cuda-#.#

1. Setting CUDA Toolkit on WSL2

$ wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin
$ sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600

$ wget https://developer.download.nvidia.com/compute/cuda/11.4.0/local_installers/cuda-repo-wsl-ubuntu-11-4-local_11.4.0-1_amd64.deb
$ sudo dpkg -i cuda-repo-wsl-ubuntu-11-4-local_11.4.0-1_amd64.deb
$ sudo apt-key add /var/cuda-repo-wsl-ubuntu-11-4-local/7fa2af80.pub

$ sudo apt-get update
$ sudo apt-get -y install cuda

설치 확인

$ cd /usr/local/cuda-11.4/samples/4_Finance/BlackScholes
$ sudo make BlackScholes
$ ./BlackScholes
[./BlackScholes] - Starting...
GPU Device 0: "Pascal" with compute capability 6.1

Initializing data...
...allocating CPU memory for options.
...allocating GPU memory for options.
...generating input data in CPU mem.
...copying input data to GPU mem.
Data init done.

Executing Black-Scholes GPU kernel (512 iterations)...
Options count             : 8000000
BlackScholesGPU() time    : 0.227898 msec
Effective memory bandwidth: 351.033566 GB/s
Gigaoptions per second    : 35.103357

BlackScholes, Throughput = 35.1034 GOptions/s, Time = 0.00023 s, Size = 8000000 options, NumDevsUsed = 1, Workgroup = 128

Reading back GPU results...
Checking the results...
...running CPU calculations.

Comparing the results...
L1 norm: 1.741792E-07
Max absolute error: 1.192093E-05

Shutting down...
...releasing GPU memory.
...releasing CPU memory.
Shutdown done.

[BlackScholes] - Test Summary

NOTE: The CUDA Samples are not meant for performance measurements. Results may vary when GPU Boost is enabled.

Test passed

Chane Repository

$ sudo nano /etc/apt/sources.list

Replace: Ctrl + \

Search (to replace): archive.ubuntu.com

Replace with: mirror.kakao.com

Save: Ctrl + s

Exit: Ctrl + x

확인

$ sudo apt update

개발환경 설정

1. PIP install

$ sudo apt-get install python3-pip
$ pip install --upgrade pip

2. Pytorch, Torchvision install

$ pip3 install torch torchvision torchaudio

3. OpenCV

$ pip install opencv-python

4. TensorRT

 - CUDA toolkit, PyCUDA

$ pip install numpy cupy

Kepler architecture 이상의 GPU 필요

- TensorRT C++

https://developer.nvidia.com/tensorrt

$ wget https://developer.nvidia.com/compute/machine-learning/tensorrt/secure/8.4.3/local_repos/nv-tensorrt-repo-ubuntu1804-cuda11.6-trt8.4.3.1-ga-20220813_1-1_amd64.deb
$ sudo dpkg -i nv-tensorrt-repo-ubuntu1804-cuda11.6-trt8.4.3.1-ga-20220813_1-1_amd64.deb
(Reading database ... 71998 files and directories currently installed.)
Preparing to unpack nv-tensorrt-repo-ubuntu1804-cuda11.6-trt8.4.3.1-ga-20220813_1-1_amd64.deb ...
Unpacking nv-tensorrt-repo-ubuntu1804-cuda11.6-trt8.4.3.1-ga-20220813 (1-1) over (1-1) ...
Setting up nv-tensorrt-repo-ubuntu1804-cuda11.6-trt8.4.3.1-ga-20220813 (1-1) ...

위와 같이 설치하면 header 파일 및 lib 파일을 찾을 수 없음.

아래 링크의 TAR Package를 받아야 함

https://developer.nvidia.com/compute/machine-learning/tensorrt/secure/8.4.3/tars/tensorrt-8.4.3.1.linux.x86_64-gnu.cuda-11.6.cudnn8.4.tar.gz

$ tar -xvf TensorRT-8.4.3.1.Linux.x86_64-gnu.cuda-11.6.cudnn8.4.tar.gz
$ mv TensorRT-8.4.3.1 ~/dev/

include, lib 경로를 환경변수에 등록

$ sudo nano ~/.bashrc

...

export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:/lib64:$LD_LIBRARY_PATH:~/dev/TensorRT-8.4.3.1/lib

bashrc 변경 적용 또는 재시작

$ source ~/.bashrc

TensorRT-8.4.3.1/python/ 폴더에서 현재 파이썬 버전에 맞는 패키지 설치

$ pip install tensorrt-8.4.3.1-cp36-none-linux_x86_64.whl
Defaulting to user installation because normal site-packages is not writeable
Processing ./tensorrt-8.4.3.1-cp36-none-linux_x86_64.whl
Installing collected packages: tensorrt
Successfully installed tensorrt-8.4.3.1

TensorRT-8.4.3.1/uff/ 폴더에서도 설치

$ cd ../uff/
$ pip install uff-0.6.9-py2.py3-none-any.whl
Defaulting to user installation because normal site-packages is not writeable
Processing ./uff-0.6.9-py2.py3-none-any.whl
Requirement already satisfied: protobuf>=3.3.0 in /home/ym/.local/lib/python3.6/site-packages (from uff==0.6.9) (3.17.3)
Requirement already satisfied: numpy>=1.11.0 in /home/ym/.local/lib/python3.6/site-packages (from uff==0.6.9) (1.19.5)
Requirement already satisfied: six>=1.9 in /home/ym/.local/lib/python3.6/site-packages (from protobuf>=3.3.0->uff==0.6.9) (1.15.0)
Installing collected packages: uff
Successfully installed uff-0.6.9

$ cd ../graphsurgeon/
$ pip install graphsurgeon-0.4.6-py2.py3-none-any.whl
Defaulting to user installation because normal site-packages is not writeable
Processing ./graphsurgeon-0.4.6-py2.py3-none-any.whl
Installing collected packages: graphsurgeon
Successfully installed graphsurgeon-0.4.6

$ cd ../onnx_graphsurgeon/
$ pip install onnx_graphsurgeon-0.3.12-py2.py3-none-any.whl
Defaulting to user installation because normal site-packages is not writeable
Processing ./onnx_graphsurgeon-0.3.12-py2.py3-none-any.whl
Collecting onnx
  Using cached onnx-1.12.0.tar.gz (10.1 MB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy in /home/ym/.local/lib/python3.6/site-packages (from onnx-graphsurgeon==0.3.12) (1.19.5)
Requirement already satisfied: protobuf<=3.20.1,>=3.12.2 in /home/ym/.local/lib/python3.6/site-packages (from onnx->onnx-graphsurgeon==0.3.12) (3.17.3)
Requirement already satisfied: typing-extensions>=3.6.2.1 in /home/ym/.local/lib/python3.6/site-packages (from onnx->onnx-graphsurgeon==0.3.12) (3.7.4.3)
Requirement already satisfied: six>=1.9 in /home/ym/.local/lib/python3.6/site-packages (from protobuf<=3.20.1,>=3.12.2->onnx->onnx-graphsurgeon==0.3.12) (1.15.0)
Building wheels for collected packages: onnx
  Building wheel for onnx (setup.py) ... error

마지막 오류 해결해야 함...

- Protobuf 설치

$ pip3 install "protobuf>=3.11.0,<=3.20.1"

여전히 오류가 나지만... python에서 import tensorrt 문제 없음

$ sudo apt-get update
...
W: GPG error: http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY A4B469963BF863CC
E: The repository 'http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

아래 사이트로 접속해 NO_PUBKEY 뒷 부분의 코드 앞에 '0x'를 붙여서 검색

MIT GPG KeyServer: http://pgp.mit.edu/ 

pub 오른쪽 key ID 항목(3BF863CC) 복사해서 키 등록

$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 3BF863CC
Executing: /tmp/apt-key-gpghome.8Pt4MWCK04/gpg.1.sh --keyserver keyserver.ubuntu.com --recv 3BF863CC
gpg: key A4B469963BF863CC: public key "cudatools <cudatools@nvidia.com>" imported
gpg: Total number processed: 1
gpg:               imported: 1

업데이트 확인

$ sudo apt-get update
Get:1 http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  InRelease [1581 B]
Hit:2 https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64  InRelease
Ign:3 http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  InRelease
Get:4 http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  Packages [950 kB]
Hit:5 http://security.ubuntu.com/ubuntu bionic-security InRelease
Hit:6 http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  Release
Hit:8 http://archive.ubuntu.com/ubuntu bionic InRelease
Hit:9 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
Hit:10 http://archive.ubuntu.com/ubuntu bionic-backports InRelease
Fetched 950 kB in 2s (582 kB/s)
Reading package lists... Done​

https://cmake.org/download/

최신 릴리즈 파일의 링크 복사

https://github.com/Kitware/CMake/releases/download/v3.24.2/cmake-3.24.2.tar.gz

$ wget https://github.com/Kitware/CMake/releases/download/v3.24.2/cmake-3.24.2.tar.gz

$ tar -xvzf cmake-3.24.2.tar.gz
$ cd cmake-3.24.2
$ ./bootstrap --prefix=/usr/local
$ make
$ make install

CMake Error at Utilities/cmcurl/CMakeLists.txt:591 (message):
  Could not find OpenSSL.  Install an OpenSSL development package or
  configure CMake with -DCMAKE_USE_OPENSSL=OFF to build without OpenSSL.

$ sudo apt-get install libssl-dev

$ openssl version -a
OpenSSL 1.1.1  11 Sep 2018
built on: Mon Jul  4 11:25:51 2022 UTC
platform: debian-amd64
options:  bn(64,64) rc4(16x,int) des(int) blowfish(ptr) 
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -Wa,--noexecstack -g -O2 -fdebug-prefix-map=/build/openssl-wL7Fqk/openssl-1.1.1=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
OPENSSLDIR: "/usr/lib/ssl"
ENGINESDIR: "/usr/lib/x86_64-linux-gnu/engines-1.1"
Seeding source: os-specific

설치 확인

$ cmake --version

명령어를 찾을 수 없다고 나오는 경우

$vi ~./bash_profile

PATH=/usr/local/bin:$PATH:$HOME/bin

ssh 재시작

아래 명령어로 소유자를 root에서 사용 계정으로 바꿔줘야 함

$ sudo chown -R jylee *