PyTorch vs TensorFlow

PyTorch and TensorFlow are two of the most prominent frameworks used in the Deep Learning community. TensorFlow was developed by the Google Brain team in 2015 whereas PyTorch was created by Facebook AI Research Lab in 2016. While both are widely used among developers, researchers and industry professionals, and have gained immense popularity, each of them have their own advantages and uniqueness over other. Let’s dive deep into the nuances of both these frameworks.

PyTorch vs TensorFlow (Differences):

1. Popularity: Currently TensorFlow is more popular among researchers because of its vast community and well written documentation. Comparatively, PyTorch is relatively new but it is gaining traction at a very fast pace. It also has nicely written documentation and the community is rapidly expanding. 

1. Graphs: TensorFlow supports static graph whereas PyTorch supports dynamic graph. In TensorFlow, the entire graph needs to be created initially and it cannot be modified in between; however in PyTorch the graph is dynamic by default. Static graph creates an issue with scenarios involving coding a Recurrent Neural Network type of algorithm in which length of input will have to remain constant which is not the case with a dynamic graph. 

1. Python Friendly: While coding in PyTorch seems more python friendly as it is closely integrated with python and the modules are encapsulated in a developer friendly manner. With TensorFlow, writing code can become a bit cumbersome because one has to write the boilerplate code of sessions, placeholders again and again. 

1. Debugging: Common python debuggers such as pdb can be used with PyTorch however with TensorFlow an option called tfdbg needs to be used to evaluate tensors at runtime. 

1. Visualization: For visualization purposes, Tensorboard is a superb tool to use with TensorFlow. It can be used to compare the training parameters or to view results of different training experiments. It can visualize the graphs of the models, display images, create histograms, and also display summaries. PyTorch does not have such an efficient tool integrated with it for visualization. Plotting libraries such as matplotlib and seaborn can be used for the same with PyTorch. 

1. Deployment: TensorFlow is truly a winner for this category. TensorFlow provides a service called TensorFlow Serving which supports deployments in house or even on cloud. It also supports deployment on a gRPC server. Additionally, TensorFlow Lite supports deployment on mobile or embedded devices. It provides a compressed and optimised version of model for better performance on edge devices. However, with PyTorch, previously only standard REST APIs and Flask framework could be used to wrap the model for deployment. Recently, PyTorch has also released TorchServe and PyTorch Live for deployment but it still does not match up to the functionalities provided by TensorFlow. 

1. Multi GPU Training: PyTorch supports DataParallel module which can wrap modules and perform training on multiple GPUs with minimal effort. TensorFlow does not have a direct functionality for data parallelism however, it can be done with manual efforts. 

1. Pretrained Models: The availability of pretrained models provided by these frameworks matter a lot for deciding which framework to use as in most cases training from scratch is computationally intensive. HuggingFace provides a lot of pretrained models in PyTorch and TensorFlow, however, the number of models it provides in PyTorch is far more than TensorFlow. Over the past two years, there has been a major shift in research papers utilizing PyTorch for development. 

PyTorch and TensorFlow both have equally strong ecosystems in terms of collaborative platforms, library integrations and deployment resources. Based on the features that are described above, let’s look at which framework outperforms other in each of these categories. 

To summarise, TensorFlow should be used if one wants to deploy models on mobile devices, wish to have good visualization capabilities, and refer to comprehensive documentation. On the other hand, PyTorch should be used for a better development and debugging experience, dynamic graph creation, and convenience for multi-gpu training.