TensorFlow

TensorFlow is an open source software library for numerical calculation based on graph modeling (data flow graphs). A graph is defined as an abstract pipeline of mathematical operations operating on tensors, and are also known as multidimensional arrays. Each graph consists of nodes and arcs, wherein the nodes are operations on the data, and the arcs represent the tensors that pass through the various operations.

TensorFlow is the most commonly used library in the field of machine learning and neural networks. It has numerous APIs, including the lowest level, that is, TensorFlow Core, allows complete control over programming. These APIs are those typically used in the field of machine learning, since they make it possible to check in detail all the elements of the model being implemented. The highest level APIs are available and built from TensorFlow Core. In some cases, they can make some operations such as repetitive and predefined tasks faster and simpler, but generally preclude the possibility of going into detail, and in the implementation of a neural network it is often necessary to have a more precise control over operations. However, they can still be useful for the development of standard machine learning models.

TensorFlow provides interfaces for different languages, including Python and C or C ++, with full support, and Go or Java in Beta. It also supports parallel computing on GPUs or CPUs, and distributed computation allows execution even on mobile devices.

A TensorFlow program is typically structured in the following two distinct phases:

• Construction: In this phase, the various operations of the graph that will be performed on the input tensors are defined.
• Execution: In this phase, the operations defined in the previous phase are evaluated so as to retrieve the numerical output. The execution of operations is managed through the session object of TensorFlow.

The fundamental unit in TensorFlow is the tensor. A tensor consists of a set of primitive type values ​​modeled as a multidimensional array. In TensorFlow, almost all the functions contained in the API take tensors as inputs and always output tensors. Each tensor contained in the graph has a unique name that can be specified by the user, or otherwise automatically assigned. 

For more information on tensors and any other resources related to TensorFlow, you can refer to the official site of the framework, as shown in the following screenshot:

The default behavior in TensorFlow is to allocate all the components (tensors and operations) in the GPU memory (if it has been installed with computational support on a GPU). However, you can manually specify where to allocate each tensor and operation. It is recommended to minimize switches between the CPU and GPU as these slow down execution.

TensorFlow also allows you to define scopes for variables through which a namespace mechanism is managed; they facilitate the definition of complex models. Scopes are also very important for sharing variables between multiple graphs.

TensorFlow provides the following two options for reading input data:

• Manual: Observations are manually read and organized in batches, and then passed to the model. It is a simple mechanism to use, but it can become very slow because the data must be continually copied from the Python environment to the TensorFlow environment.
• Integrated: All operations to read data and organize them in batches of observations are implemented within the graph. It is a less intuitive mechanism to use, but presents a large increase in performance, since all data always remains in the TensorFlow environment.

All basic operations for neural networks are implemented in TensorFlow as graph nodes. The framework automatically manages everything needed to implement the forward and backward pass, including the automatic calculation of derivatives.

Among the main operations available for the construction of neural networks models, we find the following:

• Convolutions
• Sum of the bias
• Fully connected levels
• Activation functions
• Pooling 
• Prediction functions

Finally, there is a suite of tools for graphic display that is fully integrated into TensorFlow called TensorBoard. It allows visualizing the computational graph of the model and many other statistics useful for the analysis of the training process.