Asked  7 Months ago    Answers:  5   Viewed   59 times

I am new to TensorFlow. I have recently installed it (Windows CPU version) and received the following message:

Successfully installed tensorflow-1.4.0 tensorflow-tensorboard-0.4.0rc2

Then when I tried to run

import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
sess.run(hello)
'Hello, TensorFlow!'
a = tf.constant(10)
b = tf.constant(32)
sess.run(a + b)
42
sess.close()

(which I found through https://github.com/tensorflow/tensorflow)

I received the following message:

2017-11-02 01:56:21.698935: I C:tf_jenkinshomeworkspacerel-winMwindowsPY36tensorflowcoreplatformcpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2

But when I ran

import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))

it ran as it should and output Hello, TensorFlow!, which indicates that the installation was successful indeed but there is something else that is wrong.

Do you know what the problem is and how to fix it?

 Answers

62

What is this warning about?

Modern CPUs provide a lot of low-level instructions, besides the usual arithmetic and logic, known as extensions, e.g. SSE2, SSE4, AVX, etc. From the Wikipedia:

Advanced Vector Extensions (AVX) are extensions to the x86 instruction set architecture for microprocessors from Intel and AMD proposed by Intel in March 2008 and first supported by Intel with the Sandy Bridge processor shipping in Q1 2011 and later on by AMD with the Bulldozer processor shipping in Q3 2011. AVX provides new features, new instructions and a new coding scheme.

In particular, AVX introduces fused multiply-accumulate (FMA) operations, which speed up linear algebra computation, namely dot-product, matrix multiply, convolution, etc. Almost every machine-learning training involves a great deal of these operations, hence will be faster on a CPU that supports AVX and FMA (up to 300%). The warning states that your CPU does support AVX (hooray!).

I'd like to stress here: it's all about CPU only.

Why isn't it used then?

Because tensorflow default distribution is built without CPU extensions, such as SSE4.1, SSE4.2, AVX, AVX2, FMA, etc. The default builds (ones from pip install tensorflow) are intended to be compatible with as many CPUs as possible. Another argument is that even with these extensions CPU is a lot slower than a GPU, and it's expected for medium- and large-scale machine-learning training to be performed on a GPU.

What should you do?

If you have a GPU, you shouldn't care about AVX support, because most expensive ops will be dispatched on a GPU device (unless explicitly set not to). In this case, you can simply ignore this warning by

# Just disables the warning, doesn't take advantage of AVX/FMA to run faster
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'

... or by setting export TF_CPP_MIN_LOG_LEVEL=2 if you're on Unix. Tensorflow is working fine anyway, but you won't see these annoying warnings.


If you don't have a GPU and want to utilize CPU as much as possible, you should build tensorflow from the source optimized for your CPU with AVX, AVX2, and FMA enabled if your CPU supports them. It's been discussed in this question and also this GitHub issue. Tensorflow uses an ad-hoc build system called bazel and building it is not that trivial, but is certainly doable. After this, not only will the warning disappear, tensorflow performance should also improve.

Tuesday, June 1, 2021
 
Gigamegs
answered 7 Months ago
64

Those are warnings, not errors (as indicated by the W after the colon. Errors have an E there).

The warnings refer to the fact that your CPU supports SSE Instructions, which allow some fast in-hardware-parallel operations. Enabling these operations is a compile-time operation (i.e. to use SSE you need to build the library from the source enabling the specific SSE version you're targeting), in which case you might take a look at this question.

Note, however, that SSE support influences only the computation speed. Tensorflow will work with or without SSE, but it might take longer for your code to run. Note, also, that this influences only the CPU. If you're using the GPU build of Tensorflow, all the operations run on the GPU will not benefit of SSE instructions.

Wednesday, June 30, 2021
 
Farnabaz
answered 5 Months ago
94

On linux (or unix machines) the information about your cpu is in /proc/cpuinfo. You can extract information from there by hand, or with a grep command (grep flags /proc/cpuinfo).

Also most compilers will automatically define __AVX2__ so you can check for that too.

Thursday, July 29, 2021
 
radmen
answered 4 Months ago
93

As indicated in the comments to your question, that intrinsic doesn't map to an actual AVX instruction; it is an Intel extension to the intrinsic set. The implementation likely uses many underlying instructions, as a logarithm isn't a trivial operation.

If you'd like to use a non-Intel compiler but want a fast logarithm implementation, you might check out this open-source implementation of sin(), cos(), exp(), and log() functions using AVX. They are based on an earlier SSE2 version of the same functions.

Wednesday, August 4, 2021
 
Octopus
answered 4 Months ago
95

The working example of such Dockerfile that can be used as a starting point is there: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/docker (see README.md for details).

More accurately, it is a set of parameterized Docker files, the build is started with parameterized_docker_build.sh. An example of a command that successfully compiles TensorFlow inside Docker is:

export TF_DOCKER_BUILD_IS_DEVEL=YES
export TF_DOCKER_BUILD_TYPE=CPU
export TF_DOCKER_BUILD_PYTHON_VERSION=PYTHON3
export TF_DOCKER_BUILD_DEVEL_BRANCH=master
tensorflow/tools/docker/parameterized_docker_build.sh

For the purpose of building TensorFlow with custom flags use TF_DOCKER_BUILD_IS_DEVEL=YES as non-devel Docker files just downloads precompiled Docker binaries from the server.

TensorFlow team just started to build development Docker images with AVX recently.

For SSE see this question. You can modify bazel command line in your local copy of https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/docker/Dockerfile.devel.

PS. For non-devel TensorFlow build with custom options you could look at https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/ci_build.

Monday, October 11, 2021
 
ShadowZzz
answered 2 Months ago
Only authorized users can answer the question. Please sign in first, or register a free account.
Not the answer you're looking for? Browse other questions tagged :
 
Share