[DM/DL] x86+우분투 리눅스 환경에서 인셉션v3 빌드하기 (3)

1_9. inception v3 model을 실습(이긴 한데..문제가 있다..)

이제 실습만 하면 되고 이거면 원래 쉽게 끝나야하는데..문제가 있다.
결과부터 얘기하면 텐서플로우 1.1.0이 오류 bazel의 버전과의 문제로 오류가 나는 경우가 있다. 그래서 때로는 텐서플로우 1.0.0으로 다운그레이드를 하고 실습했다.
(pip install tensorflow==1.0.0) 그리고 재미있는 게 conda와 pip버전 모두 tensorflow를 설치되어서 혼동을 준다...하나만 쓰자 ;;

즉 이런 꼴이 날수도 있다는 뜻이다..(이상한 것이 보이지 않는가?)
root@hoonjo-VirtualBox:~/inception/models/inception# conda list | grep tensor
tensorflow                1.1.0               np112py36_0  
tensorflow-gpu            1.0.0                     <pip>

그리고 만날수 있는 대표적인 오류들은 다음과 같다.
이 오류들은 경우의 수가 다양해서 모두 알순 없지만 bazel 그리고 tensorflow의 버전 별의 차이가 유력하다.

1) AttributeError: module 'tensorflow' has no attribute 'app'

2) ImportError: libcuda.so.1: cannot open shared object file: No such file or directory

Failed to load the native TensorFlow runtime.

inception 디렉터리를 만들고 환경 설정을 해준다.
root@hoonjo-VirtualBox:~# mkdir ~/inception
root@hoonjo-VirtualBox:~# cd ~/inception/
root@hoonjo-VirtualBox:~/inception# export INCEPTION_DIR=/root/inception

해당 디렉터리에 inception 모델 파일을 다운로드 하고 압축해제도 해준다.
root@hoonjo-VirtualBox:~/inception# curl -O http://download.tensorflow.org/model 
s/image/imagenet/inception-v3-2016-03-01.tar.gz
<중략>
root@hoonjo-VirtualBox:~/inception# tar -xvf inception-v3-2016-03-01.tar.gz
inception-v3/
inception-v3/checkpoint
inception-v3/README.txt
inception-v3/model.ckpt-157585
root@hoonjo-VirtualBox:~/inception# 
root@hoonjo-VirtualBox:~/inception# git clone https://github.com/tensorflow/mode 
ls.git
The program 'git' is currently not installed. You can install it by typing:
apt install git

우분투 기본 패키지에는 'git'이 없나 보다 'apt'으로 설치한다.
root@hoonjo-VirtualBox:~/inception# apt install git

다시 시작한다.
root@hoonjo-VirtualBox:~/inception# git clone https://github.com/tensorflow/mode 
ls.git
Cloning into 'models'...
remote: Counting objects: 4845, done.
remote: Compressing objects:   9% (1/11) 
<중략>

root@hoonjo-VirtualBox:~/inception# export FLOWERS_DIR=/root/inception/models/inception

root@hoonjo-VirtualBox:~/inception# which bazel
/root/anaconda3/bin/bazel
root@hoonjo-VirtualBox:~/inception# pwd
/root/inception
root@hoonjo-VirtualBox:~/inception# cd models/
root@hoonjo-VirtualBox:~/inception/models# cd inception/



캐시를 설정하고 안 하고에 따라 성능차이가 발생한다.
root@hoonjo-VirtualBox:~/inception/models/inception# bazel build
Extracting Bazel installation...
................
INFO: Found 0 targets...
INFO: Elapsed time: 18.818s, Critical Path: 0.01s

root@hoonjo-VirtualBox:~/inception/models/inception# export TEST_TMPDIR=/root/.cache 

root@hoonjo-VirtualBox:~/inception/models/inception# bazel build
INFO: $TEST_TMPDIR defined: output root default is '/root/.cache'.
...................
INFO: Found 0 targets...
INFO: Elapsed time: 9.395s, Critical Path: 0.02s
root@hoonjo-VirtualBox:~/inception/models/inception# 

여러번 다운로드 했더니 다운로드 하는건 skip해 버린다..;;

root@hoonjo-VirtualBox:~/inception/models/inception#bazel-bin/inception/download_and_preprocess_flowers $FLOWERS_DIR/data

Skipping download of flower data.

Saving results to /root/inception/models/inception/data

Determining list of input files and labels from /root/inception/models/inception/data/raw-data/validation.

Found 500 JPEG files across 5 labels inside /root/inception/models/inception/data/raw-data/validation.

Launching 2 threads for spacings: [[0, 250], [250, 500]]

W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE3 instructions, but these are available on your machine and could speed up CPU computations.

W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.

W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.

W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.

2017-06-02 08:53:31.392194 [thread 1]: Wrote 250 images to /root/inception/models/inception/data/validation-00001-of-00002

2017-06-02 08:53:31.392236 [thread 1]: Wrote 250 images to 250 shards.

2017-06-02 08:53:31.422970 [thread 0]: Wrote 250 images to /root/inception/models/inception/data/validation-00000-of-00002

2017-06-02 08:53:31.423456 [thread 0]: Wrote 250 images to 250 shards.

2017-06-02 08:53:31.602173: Finished writing all 500 images in data set.

Determining list of input files and labels from /root/inception/models/inception/data/raw-data/train.

Found 3170 JPEG files across 5 labels inside /root/inception/models/inception/data/raw-data/train.

Launching 2 threads for spacings: [[0, 1585], [1585, 3170]]

2017-06-02 08:53:38.656836 [thread 0]: Processed 1000 of 1585 images in thread batch.

2017-06-02 08:53:38.734467 [thread 1]: Processed 1000 of 1585 images in thread batch.

2017-06-02 08:53:42.872016 [thread 1]: Wrote 1585 images to /root/inception/models/inception/data/train-00001-of-00002

2017-06-02 08:53:42.872147 [thread 1]: Wrote 1585 images to 1585 shards.

2017-06-02 08:53:42.914258 [thread 0]: Wrote 1585 images to /root/inception/models/inception/data/train-00000-of-00002

2017-06-02 08:53:42.914331 [thread 0]: Wrote 1585 images to 1585 shards.

2017-06-02 08:53:43.752331: Finished writing all 3170 images in data set.

꽃 사진을 분류하는 샘플이라고 한다. 테스트를 위해서 빌드한다.
root@hoonjo-VirtualBox:~/inception/models/inception# du -sm data/raw-data/train/*
29 data/raw-data/train/daisy
43 data/raw-data/train/dandelion
1 data/raw-data/train/LICENSE.txt
33 data/raw-data/train/roses
47 data/raw-data/train/sunflowers
48 data/raw-data/train/tulips

root@hoonjo-VirtualBox:~/inception/models/inception# bazel build inception/flowers_train

INFO: $TEST_TMPDIR defined: output root default is '/root/.cache'.

...........

INFO: Found 1 target...

Target //inception:flowers_train up-to-date:

  bazel-bin/inception/flowers_train

INFO: Elapsed time: 9.900s, Critical Path: 0.04s

준비 끝!!! 그렇다..여기까지가 준비다..

참고자료 : ppc64le에서 테스트한 결과
 - http://hwengineer.blogspot.kr/2017/05/minsky-continuum-anaconda.html